-- Logs begin at Mon 2022-07-25 08:02:04 UTC, end at Mon 2022-07-25 09:11:29 UTC. -- Jul 25 08:31:15.418699 dv-cinder-850840-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 25 08:31:15.427977 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 25 08:31:15.429992 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 25 08:31:15 2022] *** Jul 25 08:31:15.429992 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 25 08:31:15.429992 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 25 08:31:15.429992 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: nodename: dv-cinder-850840-1-iscsi Jul 25 08:31:15.429992 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: machine: x86_64 Jul 25 08:31:15.429992 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: clock source: unix Jul 25 08:31:15.429992 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: pcre jit disabled Jul 25 08:31:15.429992 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: detected number of CPU cores: 4 Jul 25 08:31:15.429992 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: current working directory: / Jul 25 08:31:15.429992 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: detected binary path: /usr/bin/uwsgi-core Jul 25 08:31:15.429992 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: your processes number limit is 47680 Jul 25 08:31:15.430712 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: your memory page size is 4096 bytes Jul 25 08:31:15.430712 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: detected max file descriptor number: 2048 Jul 25 08:31:15.430712 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: lock engine: pthread robust mutexes Jul 25 08:31:15.430712 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: thunder lock: enabled Jul 25 08:31:15.430712 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 25 08:31:15.430712 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 25 08:31:15.469267 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: Python main interpreter initialized at 0x55d25cb779c0 Jul 25 08:31:15.469267 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: python threads support enabled Jul 25 08:31:15.469267 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: your server socket listen backlog is limited to 100 connections Jul 25 08:31:15.469267 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: your mercy for graceful operations on workers is 90 seconds Jul 25 08:31:15.469592 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: mapped 403077 bytes (393 KB) for 2 cores Jul 25 08:31:15.469695 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: *** Operational MODE: preforking *** Jul 25 08:31:15.469970 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: *** uWSGI is running in multiple interpreter mode *** Jul 25 08:31:15.469970 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: spawned uWSGI master process (pid: 86742) Jul 25 08:31:15.470156 dv-cinder-850840-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 25 08:31:15.476595 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: spawned uWSGI worker 1 (pid: 86751, cores: 1) Jul 25 08:31:15.476595 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: spawned uWSGI worker 2 (pid: 86752, cores: 1) Jul 25 08:31:15.476595 dv-cinder-850840-1-iscsi devstack@keystone.service[86742]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 25 08:31:20.951062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 08:31:20.951062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: from cryptography.utils import int_from_bytes Jul 25 08:31:20.951859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 08:31:20.951859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: from cryptography.utils import int_from_bytes Jul 25 08:31:20.952277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 08:31:20.952277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: from cryptography.utils import int_from_bytes Jul 25 08:31:20.952386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 08:31:20.952386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: from cryptography.utils import int_from_bytes Jul 25 08:31:21.001530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 08:31:21.001726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 08:31:21.001916 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 08:31:21.002011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] command line args: [] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 08:31:21.002123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 08:31:21.002209 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 08:31:21.002323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] command line args: [] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 08:31:21.002408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ================================================================================ {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 08:31:21.002598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] admin_token = **** {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.002683 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 08:31:21.002805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ================================================================================ {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 08:31:21.002883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] config_dir = [] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.003027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.003121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] admin_token = **** {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.003193 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] config_source = [] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.003293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] config_dir = [] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.003371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] debug = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.003488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.003574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.003816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] config_source = [] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.003885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] debug = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.003949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.004084 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.004208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.004317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.004412 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.004513 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.004666 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.004779 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.004875 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] list_limit = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.004970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] log_config_append = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.005069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.005172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.005276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] list_limit = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.005378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] log_dir = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.005480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] log_config_append = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.005583 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] log_file = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.005678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.005778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.005891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] log_dir = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.005891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] log_file = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.006073 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.006197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.006273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.006394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.006476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.006575 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.006654 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.006778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.006854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.006983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.007061 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.007188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.007269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.007416 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.007500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.007661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.007745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.007898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.008018 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.008132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.008210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.008369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.008488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.008630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.008733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.008887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.008972 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.009140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.009388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.009467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.009593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] publish_errors = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.009825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.009946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.010060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.010137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] publish_errors = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.010260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.010337 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.010477 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.010556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.010716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.010813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.010954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] standard_threads = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.011039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.011189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.011273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.011430 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] standard_threads = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.011578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.011660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.011731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.011806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] use_journal = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.011908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.011989 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] use_json = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.012091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.012170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] use_stderr = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.012265 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] use_syslog = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.012348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] use_journal = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.012442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] use_json = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.012529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.012647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] use_stderr = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.012806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.012894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] use_syslog = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.013016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.013094 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 08:31:21.013282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.013375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.013527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.013607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.013781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.013859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.014024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.014104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.014277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.014355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.014519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.014596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.014768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.014847 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.015009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.015087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.015262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.015344 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.015501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.015581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.015754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.015837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.015992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.016074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.016242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.016328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.016486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.016589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.016747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.016825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.017008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.017090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.017252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.017329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.017505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.017581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.017745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.017832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.018003 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.018080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.018247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.018332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.018507 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.018598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.018761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.018840 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.018997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.019110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.019232 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.019318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.019467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.019551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.019708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.019792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.019942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.020026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.020184 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.020262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.020421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.020510 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.020679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.020761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.020930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.021006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.021169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.021264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.021407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.021486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.021660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.021738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.021894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.021985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.022134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.022212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.022368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.022445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.022600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.022752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.022827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.022896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.023055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.023133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.023288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.023374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.023522 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.023599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.023759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.023837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.023992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.024077 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.024230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.024308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.024465 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.024589 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.024727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.024844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.024964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.025083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.025232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.025343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.025461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.025547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.025734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.025878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.025961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.026033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.026202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.026297 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.026441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.026564 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.026684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.026827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.026908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.026993 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.027150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.027258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.027387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.027464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.027610 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.027702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.027847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.027940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.028088 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.028173 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.028330 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.028417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.028561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.028687 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.028822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.028938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.029064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.029150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.029303 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.029393 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.029547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.029626 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.029803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.029884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.030050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.030140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.030291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.030370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.030530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.030608 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.030770 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.030860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.031011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.031089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.031250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.031333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.031491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.031572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.031733 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.031810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.031969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.032052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.032207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.032294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.032729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.032812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.032969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.033059 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.033207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.033289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.033446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.033524 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.033689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.033767 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.033938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.034018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.034177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.034256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.034425 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.034508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.034660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.034739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.034888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.034973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.035126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.035213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.035360 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.035449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.035593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.035674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.035833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.035912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.036068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.036152 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.036300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.036380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.036543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.036666 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.036805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.036883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.037053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.037134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.037291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.037371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.037546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.037623 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.037790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.037871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.038037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.038114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.038274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] auth.external = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.038353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.038511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.038591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.038764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] auth.external = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.038841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.039002 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.039111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.039261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] auth.password = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.039342 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.039497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] auth.token = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.039579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.039739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.039818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] auth.password = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.039982 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.040069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] auth.token = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.040226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.040306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.040459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.040540 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.040736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.041323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.041491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.041570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.041726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.041805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.041966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] credential.caching = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.042055 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.042212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.042297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.042455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] credential.caching = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.042536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.042706 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.042793 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.042947 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.043027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.043194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.043275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.043433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.043520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.043683 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.043763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.043925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.044004 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.044169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.044261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.044433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.044525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.044696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.044778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.044959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.045055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.045233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.045313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.045482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.045565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.045746 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.045834 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.045989 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] federation.caching = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.046078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.046237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.046323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.046483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] federation.caching = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.046571 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.046728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.046808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.046961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.047039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.047215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.047304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.047451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.047530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.047705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.047785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.047943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.048029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.048201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.048280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.048438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.048517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.048711 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.048805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.048955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.049033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.caching = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.049207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.049286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.049447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.caching = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.049602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.049678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.049756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.049917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.049997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.050169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.050250 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.050407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.050486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.050647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.050727 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.050888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.050968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.051133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.051212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.051364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.051442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.051599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.051681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.051832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.051912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.052082 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.052188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.052320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.052405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.052588 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.052671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.052849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.052927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.053101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.053182 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.053362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.053441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.053602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.053680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.053843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.053924 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.054090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.054168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.054318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.054406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.054548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.054628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.054786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.054864 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.055029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.055106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.055267 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.055346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.055511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.055587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.055744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.055820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.055998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.056076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.056236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.056313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.057063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.057156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.057325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.057412 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.057569 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.057647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.057837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.057914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.058082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.058162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.058318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.058394 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.058557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.058633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.058790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.058869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.059032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.059108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.059269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.059346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.059506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.059585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.059743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.059820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.059985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.060071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.060222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.060300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.060456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.060539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.060731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.060809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.060969 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.061048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.061208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.061291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.061448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.061526 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.061687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.061772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.061923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.062000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.062161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.062237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.062399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.062478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.062641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.062718 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.062891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.062976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.063127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.063210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.063378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.063455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.063615 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.063701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.063849 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.063927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.064089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.064166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.064321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.064407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.064555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.064647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.064810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.064898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.065054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.065143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.065293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.065404 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.065546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.065661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.065799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.065914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.066051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.066169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.066405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.066483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.066653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.066738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.066889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.066965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.067125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.067202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.067367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.067453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.067606 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.067684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.067848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.067933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.068088 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.068174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.068325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.068406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.068593 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.068674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.068833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.068922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.069082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.069161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.069318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.069395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.069555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.069640 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.069796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.069874 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.070033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.070110 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.070276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.070362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.070510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.070587 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.070749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.070827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.070986 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.071070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] resource.caching = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.071227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.071311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.071474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] resource.caching = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.071552 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.071710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.071796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.071964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.072058 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.072229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.072307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.072473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.072560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.072749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.072828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.072989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.073066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.073227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.073385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.073460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] role.caching = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.073531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.073702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.073779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] role.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.073931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.074017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] role.caching = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.074183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] role.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.074260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.074418 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.074496 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.074671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.074756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.074922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.075000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.075170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.075247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.075421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.075499 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.075670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.075755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.075919 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.075997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.076151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.076234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.076408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.076486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.076667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.076748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.076927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.077178 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.077256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.077431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.077573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.077690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.077813 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.077944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.078068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.078321 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.078398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.078574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.078661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.078827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.078905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.079067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.079146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.079320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.079405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.079572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.079649 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.079809 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.079886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.080068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.080162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.080275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.080355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.080509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.080613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.080787 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.080878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.081025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.081103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.081263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.081347 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.081512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.081591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.081752 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.081829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.081986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.082065 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.082226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] token.caching = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.082305 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.082465 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.082543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.082700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.082776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.082941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] token.caching = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.083021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.083184 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.083262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.083435 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.083512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.083675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.083759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.083918 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.083995 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.084163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.084241 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.084404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.084482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.084668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.084749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.084910 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.084988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.085146 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.085226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.085392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.085469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.085638 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.085716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.085867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.085945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.086111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.086189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.086363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.086441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.086598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.086678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.connection = **** {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.086846 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.086933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.087135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.087213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.087376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.087455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.connection = **** {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.087621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.087712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.087887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.087965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.088127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.088205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.088366 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.088452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.088624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.088705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.088871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.088948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.089108 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.089196 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.089347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.089424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.089584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.089661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.089829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.089914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.090071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.090149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.090310 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.090387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.090541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.090627 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.090785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.090864 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.091023 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.091102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.091196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86751) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 08:31:21.091284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.091491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.091734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 08:31:21.091908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86752) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 08:31:21.120757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.120991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.121224 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.121558 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.121800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.122037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.129536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.130369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.159542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.159775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.160856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.161099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.211300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.212048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.261310 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.261545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.262479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.262721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.325792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.325912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.326031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.326119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.326260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.326347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.326485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.326570 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.326708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.326801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.326929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.327013 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.350268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.350505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.350608 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.350741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.350825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.350974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.351058 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.351206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.351291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.351425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.351508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.351646 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.351728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.351863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.363151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.363678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.375740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.375993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.376210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.376295 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.376438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.376522 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.376688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.376792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.376913 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.376997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.377133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86751) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.377217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.377399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.377625 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86752) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 08:31:21.394262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.394476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 08:31:21.394816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.394901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.395020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 08:31:21.395105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 08:31:21.395291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.395377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.395555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.395637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 08:31:21.395815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.395899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.396071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.396151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.396332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.396611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.407709 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.407988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.408257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.408339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.408537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.408665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.408836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.409126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.416888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.417534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.420171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.420454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.420766 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.420850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.421051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.421146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.421335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.421417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.421599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.421680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.421867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.421951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.422139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.422220 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.422407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.422487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.422680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.422761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.422945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.423025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.423216 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.423485 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.452098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.452719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.453745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.454028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.454345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.454626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.459295 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.459687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.459925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.460007 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.460322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.460632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.467333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.467573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.468002 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.468284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.472063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.472351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.472657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.472825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.472943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.473113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.473222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.473400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.473503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.473688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.473777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.473970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.474051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.474249 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.474521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.474790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.492806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.493082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.494524 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.494808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.498211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.499510 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.501906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.502308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.503204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.503609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.507343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.508686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.511584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.512891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.515251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.515553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.515845 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.516131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.516409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.516539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.516720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.516867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.517156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.517454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.517739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.518040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.536063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.536346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.536647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.536923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.537816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.538109 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.538387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.538651 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.545370 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.547063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.547148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.547339 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.548886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.549169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.550528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.550815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.551104 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.551390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.551670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.552345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.552656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.552946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.553229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.553508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.565640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.565931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.566217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.566511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.566888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.567169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.567458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.569786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.570073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.570364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.570650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.571028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.571311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.571602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.585269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.588542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.589835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.591876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.593138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.593665 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.595312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.595596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.595880 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.596468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.598251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.599916 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.600206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.600493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.603181 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.605347 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.608364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.608544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.608866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.609269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.609568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.610011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.613275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.613568 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.613961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.614263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.618632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.619524 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.619812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.623944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.624232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.624512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.627663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86751) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.627943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.628226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.628505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.628804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.629189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.629469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.629749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.630031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.630304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.630586 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.630851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.631137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.631400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86751) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.632390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86752) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 08:31:21.632710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.632997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.633278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.633555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.633930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.634209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.634523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.634817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.635102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.635388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.635656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.635940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.636212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86752) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 08:31:21.670848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d25cb779c0 pid: 86751 (default app) Jul 25 08:31:21.674252 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696d30b-b60b-435a-93cd-b92f51c5dbde None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:21.674548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696d30b-b60b-435a-93cd-b92f51c5dbde None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:21.674659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d25cb779c0 pid: 86752 (default app) Jul 25 08:31:21.674828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696d30b-b60b-435a-93cd-b92f51c5dbde None None] PATH_INFO: `/v3` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:21.679038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Mon Jul 25 08:31:21 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:29.195877 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669d10f-7630-4c10-b338-bcc19eb8e5dd None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:29.196441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669d10f-7630-4c10-b338-bcc19eb8e5dd None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:29.196441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669d10f-7630-4c10-b338-bcc19eb8e5dd None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:29.201364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:29.206426 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d36b9-81d0-4b70-9536-f7d8598737d5 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:29.206806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d36b9-81d0-4b70-9536-f7d8598737d5 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:29.206981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d36b9-81d0-4b70-9536-f7d8598737d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:29.277999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: INFO dbcounter [None req-6f5d36b9-81d0-4b70-9536-f7d8598737d5 None None] Registered counter for database keystone Jul 25 08:31:29.299280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG oslo_db.sqlalchemy.engines [None req-6f5d36b9-81d0-4b70-9536-f7d8598737d5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86751) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 08:31:29.305532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writer thread running {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 08:31:29.479803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG passlib.handlers.bcrypt [None req-6f5d36b9-81d0-4b70-9536-f7d8598737d5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86751) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 08:31:29.480146 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG passlib.handlers.bcrypt [None req-6f5d36b9-81d0-4b70-9536-f7d8598737d5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86751) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 08:31:29.534402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-6f5d36b9-81d0-4b70-9536-f7d8598737d5 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:29.536010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-6f5d36b9-81d0-4b70-9536-f7d8598737d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:29.658581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 08:31:29.806989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669d10f-7630-4c10-b338-bcc19eb8e5dd None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:29.876538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: INFO dbcounter [None req-a669d10f-7630-4c10-b338-bcc19eb8e5dd None None] Registered counter for database keystone Jul 25 08:31:29.895868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG oslo_db.sqlalchemy.engines [None req-a669d10f-7630-4c10-b338-bcc19eb8e5dd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86752) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 08:31:29.902788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writer thread running {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 08:31:30.025559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669d10f-7630-4c10-b338-bcc19eb8e5dd None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:30.029853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64257897-57e3-49e0-991c-44498981f7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:30.030517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64257897-57e3-49e0-991c-44498981f7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:30.030802 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64257897-57e3-49e0-991c-44498981f7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:30.031081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64257897-57e3-49e0-991c-44498981f7b2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:30.039561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64257897-57e3-49e0-991c-44498981f7b2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:30.091206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: WARNING keystone.common.rbac_enforcer.enforcer [None req-64257897-57e3-49e0-991c-44498981f7b2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 08:31:30.091492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64257897-57e3-49e0-991c-44498981f7b2 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:30.101837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Mon Jul 25 08:31:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 08:31:30.108309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d36b9-81d0-4b70-9536-f7d8598737d5 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:30.122821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d36b9-81d0-4b70-9536-f7d8598737d5 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:30.127245 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b30f69-e97c-45a9-95df-cc1fce1c0915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:30.128373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b30f69-e97c-45a9-95df-cc1fce1c0915 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:30.128690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b30f69-e97c-45a9-95df-cc1fce1c0915 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:30.128972 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b30f69-e97c-45a9-95df-cc1fce1c0915 admin admin] PATH_INFO: `/v3/projects/e86d2d432f134fa4aeaa99c3935e4c0c` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:30.130484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b30f69-e97c-45a9-95df-cc1fce1c0915 admin admin] RBAC: Authorizing `identity:get_project(project_id=e86d2d432f134fa4aeaa99c3935e4c0c)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:30.183883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: WARNING keystone.common.rbac_enforcer.enforcer [None req-58b30f69-e97c-45a9-95df-cc1fce1c0915 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 08:31:30.184231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b30f69-e97c-45a9-95df-cc1fce1c0915 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:30.188843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Mon Jul 25 08:31:30 2022] GET /identity/v3/projects/e86d2d432f134fa4aeaa99c3935e4c0c => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:32.154535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8595a-ada8-4092-8dc9-cc9bb16064c1 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:32.155077 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8595a-ada8-4092-8dc9-cc9bb16064c1 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:32.155077 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8595a-ada8-4092-8dc9-cc9bb16064c1 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:32.159542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:32.164923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4038ac-2265-4f54-8c7e-2b1d3484efba None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:32.165363 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4038ac-2265-4f54-8c7e-2b1d3484efba None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:32.165486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4038ac-2265-4f54-8c7e-2b1d3484efba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:32.185376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-fe4038ac-2265-4f54-8c7e-2b1d3484efba None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:32.186897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-fe4038ac-2265-4f54-8c7e-2b1d3484efba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:32.194855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 08:31:32.217439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8595a-ada8-4092-8dc9-cc9bb16064c1 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:32.229504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8595a-ada8-4092-8dc9-cc9bb16064c1 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:32.233789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6f1d0-1fca-4348-9cde-34efa8e706aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:32.234427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6f1d0-1fca-4348-9cde-34efa8e706aa admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:32.234717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6f1d0-1fca-4348-9cde-34efa8e706aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:32.234998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6f1d0-1fca-4348-9cde-34efa8e706aa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:32.242715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6f1d0-1fca-4348-9cde-34efa8e706aa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:32.244508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6f1d0-1fca-4348-9cde-34efa8e706aa admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:32.253319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Mon Jul 25 08:31:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:31:32.258335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4038ac-2265-4f54-8c7e-2b1d3484efba None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:32.267779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4038ac-2265-4f54-8c7e-2b1d3484efba None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:32.272169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f0a22-e3fb-4f02-a5b6-9f90af6444fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:32.273066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0a22-e3fb-4f02-a5b6-9f90af6444fd admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:32.273354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0a22-e3fb-4f02-a5b6-9f90af6444fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:32.273632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0a22-e3fb-4f02-a5b6-9f90af6444fd admin admin] PATH_INFO: `/v3/users/fcb4950bc93048a891412be785f132b7` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:32.275641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f0a22-e3fb-4f02-a5b6-9f90af6444fd admin admin] RBAC: Authorizing `identity:get_user(user_id=fcb4950bc93048a891412be785f132b7)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:32.277510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f0a22-e3fb-4f02-a5b6-9f90af6444fd admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:32.279078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Mon Jul 25 08:31:32 2022] GET /identity/v3/users/fcb4950bc93048a891412be785f132b7 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:34.310848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2f9f4-54ed-4d73-9347-a3addfebff4e None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:34.311273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2f9f4-54ed-4d73-9347-a3addfebff4e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:34.311410 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2f9f4-54ed-4d73-9347-a3addfebff4e None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:34.312982 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:34.317995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa817524-7e88-44e4-af83-4c6b7becc228 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:34.318433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa817524-7e88-44e4-af83-4c6b7becc228 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:34.318564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa817524-7e88-44e4-af83-4c6b7becc228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:34.322659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d14ac8-19d3-43d5-a461-edca002c4b92 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:34.322947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d14ac8-19d3-43d5-a461-edca002c4b92 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:34.323226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d14ac8-19d3-43d5-a461-edca002c4b92 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:34.324493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:31:34.329377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4be15-7b18-45c3-9eff-265937a5b642 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:34.329667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4be15-7b18-45c3-9eff-265937a5b642 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:34.329947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4be15-7b18-45c3-9eff-265937a5b642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:34.336796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-aa817524-7e88-44e4-af83-4c6b7becc228 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:34.337993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-aa817524-7e88-44e4-af83-4c6b7becc228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:34.373069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG passlib.handlers.bcrypt [None req-dac4be15-7b18-45c3-9eff-265937a5b642 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86752) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 08:31:34.373505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG passlib.handlers.bcrypt [None req-dac4be15-7b18-45c3-9eff-265937a5b642 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86752) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 08:31:34.418241 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:34.418953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-dac4be15-7b18-45c3-9eff-265937a5b642 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:34.420544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-dac4be15-7b18-45c3-9eff-265937a5b642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:34.441098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa817524-7e88-44e4-af83-4c6b7becc228 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:34.464253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:34.519885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:31:34.542201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4be15-7b18-45c3-9eff-265937a5b642 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:34.542369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa817524-7e88-44e4-af83-4c6b7becc228 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:34.575930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624b887-ed79-467c-88c6-0cfcc64f005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:34.576691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624b887-ed79-467c-88c6-0cfcc64f005e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:34.576980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624b887-ed79-467c-88c6-0cfcc64f005e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:34.577259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624b887-ed79-467c-88c6-0cfcc64f005e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:34.588377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624b887-ed79-467c-88c6-0cfcc64f005e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-5624b887-ed79-467c-88c6-0cfcc64f005e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:34.599544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:34.600622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:31:34.601604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:31:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:31:34.607106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624b887-ed79-467c-88c6-0cfcc64f005e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:34.650453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4be15-7b18-45c3-9eff-265937a5b642 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:34.652822 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:34.689873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:34.690063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132bc48d-0545-4260-a7c7-d53ab15cb427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:34.690950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bc48d-0545-4260-a7c7-d53ab15cb427 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:34.691444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bc48d-0545-4260-a7c7-d53ab15cb427 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:34.691754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bc48d-0545-4260-a7c7-d53ab15cb427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:34.707524 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624b887-ed79-467c-88c6-0cfcc64f005e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:34.741868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a086c-1b39-4e48-9531-cfea7a6312f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:34.742543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a086c-1b39-4e48-9531-cfea7a6312f7 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:34.742831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a086c-1b39-4e48-9531-cfea7a6312f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:34.743140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a086c-1b39-4e48-9531-cfea7a6312f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:34.743895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a086c-1b39-4e48-9531-cfea7a6312f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:34.744204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a086c-1b39-4e48-9531-cfea7a6312f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:34.745639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a086c-1b39-4e48-9531-cfea7a6312f7 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:34.757098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:31:34.758455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132bc48d-0545-4260-a7c7-d53ab15cb427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:34.759972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132bc48d-0545-4260-a7c7-d53ab15cb427 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:34.762617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a086c-1b39-4e48-9531-cfea7a6312f7 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:34.860043 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a086c-1b39-4e48-9531-cfea7a6312f7 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:34.860212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:31:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 08:31:34.865492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132bc48d-0545-4260-a7c7-d53ab15cb427 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:34.868187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:34.893646 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf98bf-8320-4418-ad64-27ed8b09facb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:34.894516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf98bf-8320-4418-ad64-27ed8b09facb None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:34.894798 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf98bf-8320-4418-ad64-27ed8b09facb None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:34.895072 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf98bf-8320-4418-ad64-27ed8b09facb None admin] PATH_INFO: `/v3/users/fcb4950bc93048a891412be785f132b7` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:34.896267 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf98bf-8320-4418-ad64-27ed8b09facb None admin] RBAC: Authorizing `identity:get_user(user_id=fcb4950bc93048a891412be785f132b7)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:34.897754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf98bf-8320-4418-ad64-27ed8b09facb None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:34.899899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:34 2022] GET /identity/v3/users/fcb4950bc93048a891412be785f132b7 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:31:34.905329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf98bf-8320-4418-ad64-27ed8b09facb None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:34.920239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:34.965061 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132bc48d-0545-4260-a7c7-d53ab15cb427 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:34.999057 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510025ea-444e-46e6-b7a7-e9f45060e948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:34.999852 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510025ea-444e-46e6-b7a7-e9f45060e948 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:35.000134 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510025ea-444e-46e6-b7a7-e9f45060e948 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:35.000408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510025ea-444e-46e6-b7a7-e9f45060e948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:35.004291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf98bf-8320-4418-ad64-27ed8b09facb None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:35.019572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510025ea-444e-46e6-b7a7-e9f45060e948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:35.021201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510025ea-444e-46e6-b7a7-e9f45060e948 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:35.038136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367cbd9-cdf0-443d-9537-b645771b9740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:35.038928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367cbd9-cdf0-443d-9537-b645771b9740 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:35.039210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367cbd9-cdf0-443d-9537-b645771b9740 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:35.039485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367cbd9-cdf0-443d-9537-b645771b9740 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:35.040724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367cbd9-cdf0-443d-9537-b645771b9740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:35.042162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367cbd9-cdf0-443d-9537-b645771b9740 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-510025ea-444e-46e6-b7a7-e9f45060e948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:31:35.043302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:35.044103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:35.044932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:31:35.044932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:31:35.044932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:31:35.044932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:31:35.045154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:31:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:31:35.045219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:31:35.049458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367cbd9-cdf0-443d-9537-b645771b9740 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:35.050947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510025ea-444e-46e6-b7a7-e9f45060e948 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:35.107706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:35.154300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510025ea-444e-46e6-b7a7-e9f45060e948 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:35.191000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e07cb-8906-4c21-b2d2-b377d22ed502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:35.191689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e07cb-8906-4c21-b2d2-b377d22ed502 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:35.191980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e07cb-8906-4c21-b2d2-b377d22ed502 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:35.192262 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e07cb-8906-4c21-b2d2-b377d22ed502 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:35.193060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e07cb-8906-4c21-b2d2-b377d22ed502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:35.193375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e07cb-8906-4c21-b2d2-b377d22ed502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:35.194801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e07cb-8906-4c21-b2d2-b377d22ed502 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:35.215447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 10/20] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:31:35.236261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:35.258070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367cbd9-cdf0-443d-9537-b645771b9740 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:35.296723 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb497f0-c1c9-441a-9d40-a205d5634e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:35.297420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb497f0-c1c9-441a-9d40-a205d5634e05 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:35.297693 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb497f0-c1c9-441a-9d40-a205d5634e05 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:35.297973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb497f0-c1c9-441a-9d40-a205d5634e05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:35.298834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb497f0-c1c9-441a-9d40-a205d5634e05 None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.domain.id=default, user.id=fcb4950bc93048a891412be785f132b7) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:35.299156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb497f0-c1c9-441a-9d40-a205d5634e05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:35.300680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb497f0-c1c9-441a-9d40-a205d5634e05 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:35.308829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 11/21] 10.222.0.52 () {56 vars in 1322 bytes} [Mon Jul 25 08:31:35 2022] GET /identity/v3/role_assignments?user.id=fcb4950bc93048a891412be785f132b7&scope.domain.id=default&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 233 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:37.757752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccba25-0207-4a18-af91-4b48eb59cc64 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:37.758817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccba25-0207-4a18-af91-4b48eb59cc64 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:37.759322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccba25-0207-4a18-af91-4b48eb59cc64 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:37.761220 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 11/22] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:37.766895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ef210-770f-4249-81e6-0ad9bcd900d8 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:37.767726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ef210-770f-4249-81e6-0ad9bcd900d8 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:37.768211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ef210-770f-4249-81e6-0ad9bcd900d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:37.793642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-b39ef210-770f-4249-81e6-0ad9bcd900d8 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:37.794860 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b39ef210-770f-4249-81e6-0ad9bcd900d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:37.863190 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f797c-287f-460a-b4cb-4875055a4b84 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:37.863832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f797c-287f-460a-b4cb-4875055a4b84 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:37.864333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f797c-287f-460a-b4cb-4875055a4b84 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:37.866236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 12/23] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:37.871611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6804e4c-b2ab-4b87-8cf5-b7e8bc425865 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:37.872163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6804e4c-b2ab-4b87-8cf5-b7e8bc425865 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:37.872689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6804e4c-b2ab-4b87-8cf5-b7e8bc425865 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:37.874192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 13/24] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:37.875553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd93ae9-ad45-4b6c-a812-eca479f52bd5 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:37.876056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd93ae9-ad45-4b6c-a812-eca479f52bd5 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:37.876541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd93ae9-ad45-4b6c-a812-eca479f52bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:37.888842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 12/25] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:31:37.888842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000c02b-2bef-4d83-b72e-f2a87d76a1d9 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:37.888842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000c02b-2bef-4d83-b72e-f2a87d76a1d9 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:37.888842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000c02b-2bef-4d83-b72e-f2a87d76a1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:37.920453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7cd93ae9-ad45-4b6c-a812-eca479f52bd5 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:37.922027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7cd93ae9-ad45-4b6c-a812-eca479f52bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:37.926063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-b000c02b-2bef-4d83-b72e-f2a87d76a1d9 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:37.927461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b000c02b-2bef-4d83-b72e-f2a87d76a1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:38.040779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 13/26] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:38.042491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000c02b-2bef-4d83-b72e-f2a87d76a1d9 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:38.049466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 14/27] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:31:38.049687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:38.074046 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd93ae9-ad45-4b6c-a812-eca479f52bd5 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:38.164944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:38.245698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000c02b-2bef-4d83-b72e-f2a87d76a1d9 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:38.255515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd93ae9-ad45-4b6c-a812-eca479f52bd5 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:38.283281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26497586-aa9e-44b4-b7a8-e8eba7b28f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:38.284313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26497586-aa9e-44b4-b7a8-e8eba7b28f0c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:38.284764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26497586-aa9e-44b4-b7a8-e8eba7b28f0c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:38.285312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26497586-aa9e-44b4-b7a8-e8eba7b28f0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:38.294348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a76ffc-f66d-42d2-af58-f2051abb890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:38.295095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a76ffc-f66d-42d2-af58-f2051abb890c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:38.295386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a76ffc-f66d-42d2-af58-f2051abb890c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:38.295672 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a76ffc-f66d-42d2-af58-f2051abb890c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:38.304149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26497586-aa9e-44b4-b7a8-e8eba7b28f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:38.306844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a76ffc-f66d-42d2-af58-f2051abb890c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:38.308332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a76ffc-f66d-42d2-af58-f2051abb890c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-26497586-aa9e-44b4-b7a8-e8eba7b28f0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:38.313459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:38.314287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:38.315013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:38.315013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:38.315013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:38.315013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:38.315013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:38.315013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:38.315013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:38.315013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:38.315013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:38.315013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:38.315013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:31:38.315013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:31:38.317543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 14/28] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:31:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:38.318470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26497586-aa9e-44b4-b7a8-e8eba7b28f0c None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-17a76ffc-f66d-42d2-af58-f2051abb890c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:31:38.333412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:38.334244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:38.335025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:31:38.335025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:31:38.335025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:31:38.335025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:31:38.335025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 15/29] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:38.341396 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aba60-f279-4fc3-a935-166e3db3afc2 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:38.341893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aba60-f279-4fc3-a935-166e3db3afc2 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:38.342420 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aba60-f279-4fc3-a935-166e3db3afc2 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:38.344201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 16/30] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:38.345214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aba60-f279-4fc3-a935-166e3db3afc2 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:38.407337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:38.455983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26497586-aa9e-44b4-b7a8-e8eba7b28f0c None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:38.463872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:38.470312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aba60-f279-4fc3-a935-166e3db3afc2 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:38.501677 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be895c8-dee1-4a4e-abce-33626e70e563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:38.502965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be895c8-dee1-4a4e-abce-33626e70e563 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:38.503689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be895c8-dee1-4a4e-abce-33626e70e563 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:38.504189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be895c8-dee1-4a4e-abce-33626e70e563 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:38.505835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be895c8-dee1-4a4e-abce-33626e70e563 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:38.508104 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be895c8-dee1-4a4e-abce-33626e70e563 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:38.513379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946799f-1323-4115-9bcc-cf21ffe2162f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:38.514241 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946799f-1323-4115-9bcc-cf21ffe2162f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:38.514663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946799f-1323-4115-9bcc-cf21ffe2162f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:38.515098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946799f-1323-4115-9bcc-cf21ffe2162f None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:38.516477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946799f-1323-4115-9bcc-cf21ffe2162f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:38.517056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946799f-1323-4115-9bcc-cf21ffe2162f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:38.518357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946799f-1323-4115-9bcc-cf21ffe2162f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:38.536370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 17/31] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:38.536370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946799f-1323-4115-9bcc-cf21ffe2162f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:38.635070 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946799f-1323-4115-9bcc-cf21ffe2162f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:38.644595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 15/32] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Jul 25 08:31:38 2022] POST /identity/v3/roles => generated 219 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:31:38.644595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12a044-4020-4e91-a165-2c5620785905 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:38.644595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12a044-4020-4e91-a165-2c5620785905 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:38.644595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12a044-4020-4e91-a165-2c5620785905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:38.659005 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-eb12a044-4020-4e91-a165-2c5620785905 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:38.660777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-eb12a044-4020-4e91-a165-2c5620785905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:38.694398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155482b-ba10-45f1-a676-a7c20ae5e64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:38.695473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155482b-ba10-45f1-a676-a7c20ae5e64b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:38.695977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155482b-ba10-45f1-a676-a7c20ae5e64b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:38.697101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155482b-ba10-45f1-a676-a7c20ae5e64b None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:38.698555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155482b-ba10-45f1-a676-a7c20ae5e64b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:38.699290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155482b-ba10-45f1-a676-a7c20ae5e64b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:38.701125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155482b-ba10-45f1-a676-a7c20ae5e64b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:38.717805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 18/33] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:38.719957 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b5ebb-1bb5-4e87-8c1f-489fe53bd362 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:38.720429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b5ebb-1bb5-4e87-8c1f-489fe53bd362 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:38.720913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b5ebb-1bb5-4e87-8c1f-489fe53bd362 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:38.723289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 19/34] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:38.724295 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b5ebb-1bb5-4e87-8c1f-489fe53bd362 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:38.760600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 16/35] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:38.760600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12a044-4020-4e91-a165-2c5620785905 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:38.843133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:38.883887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:38.901435 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12a044-4020-4e91-a165-2c5620785905 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:38.942179 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e990f91-b1c4-4a39-9c20-9788bdada946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:38.943084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e990f91-b1c4-4a39-9c20-9788bdada946 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:38.943378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e990f91-b1c4-4a39-9c20-9788bdada946 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:38.943665 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e990f91-b1c4-4a39-9c20-9788bdada946 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:38.944396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e990f91-b1c4-4a39-9c20-9788bdada946 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:38.946078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e990f91-b1c4-4a39-9c20-9788bdada946 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:38.970429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b5ebb-1bb5-4e87-8c1f-489fe53bd362 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:38.984138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 17/36] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:31:38 2022] POST /identity/v3/projects => generated 301 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:31:38.984138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b8c75-4d41-485b-8204-55bc964eb6e0 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:38.984138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b8c75-4d41-485b-8204-55bc964eb6e0 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:38.984138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b8c75-4d41-485b-8204-55bc964eb6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:39.000157 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-fb2b8c75-4d41-485b-8204-55bc964eb6e0 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:39.001676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-fb2b8c75-4d41-485b-8204-55bc964eb6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:39.026017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2dffd-aff3-48a6-9441-bd02a430284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:39.027143 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2dffd-aff3-48a6-9441-bd02a430284b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:39.027492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2dffd-aff3-48a6-9441-bd02a430284b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:39.027845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2dffd-aff3-48a6-9441-bd02a430284b None admin] PATH_INFO: `/v3/users/fcb4950bc93048a891412be785f132b7` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:39.029689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2dffd-aff3-48a6-9441-bd02a430284b None admin] RBAC: Authorizing `identity:get_user(user_id=fcb4950bc93048a891412be785f132b7)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:39.031274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2dffd-aff3-48a6-9441-bd02a430284b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:39.033561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 20/37] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:38 2022] GET /identity/v3/users/fcb4950bc93048a891412be785f132b7 => generated 247 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:39.038707 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2dffd-aff3-48a6-9441-bd02a430284b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:39.092672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 18/38] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:39.097629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b8c75-4d41-485b-8204-55bc964eb6e0 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:39.151132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:39.166235 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:39.184098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2dffd-aff3-48a6-9441-bd02a430284b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:39.224116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801aa10-af14-438b-a6b7-306dd31cd46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:39.224818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801aa10-af14-438b-a6b7-306dd31cd46f None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:39.225108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801aa10-af14-438b-a6b7-306dd31cd46f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:39.225389 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801aa10-af14-438b-a6b7-306dd31cd46f None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:39.226144 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801aa10-af14-438b-a6b7-306dd31cd46f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:39.226927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b8c75-4d41-485b-8204-55bc964eb6e0 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:39.228130 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801aa10-af14-438b-a6b7-306dd31cd46f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:39.261845 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b759a35-e422-486e-a17a-ae44d4b5cd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:39.262650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b759a35-e422-486e-a17a-ae44d4b5cd7c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:39.262940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b759a35-e422-486e-a17a-ae44d4b5cd7c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:39.263249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b759a35-e422-486e-a17a-ae44d4b5cd7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:39.264453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b759a35-e422-486e-a17a-ae44d4b5cd7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:39.265957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b759a35-e422-486e-a17a-ae44d4b5cd7c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:39.267602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 19/39] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:31:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:39.268351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b759a35-e422-486e-a17a-ae44d4b5cd7c None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:39.348152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 21/40] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Jul 25 08:31:39 2022] POST /identity/v3/roles => generated 225 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:31:39.349707 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098b9b8-6782-49cd-86ac-68c2e3c891f0 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:39.350004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098b9b8-6782-49cd-86ac-68c2e3c891f0 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:39.350305 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098b9b8-6782-49cd-86ac-68c2e3c891f0 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:39.351732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 22/41] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:39.352942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098b9b8-6782-49cd-86ac-68c2e3c891f0 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:39.382067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b759a35-e422-486e-a17a-ae44d4b5cd7c None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:39.385802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:39.424060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb431742-0ff4-4752-9d92-acdc32ed5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:39.424700 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb431742-0ff4-4752-9d92-acdc32ed5306 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:39.424993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb431742-0ff4-4752-9d92-acdc32ed5306 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:39.425279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb431742-0ff4-4752-9d92-acdc32ed5306 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:39.426063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb431742-0ff4-4752-9d92-acdc32ed5306 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:39.427500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb431742-0ff4-4752-9d92-acdc32ed5306 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:39.443214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 20/42] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Jul 25 08:31:39 2022] POST /identity/v3/roles => generated 223 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:31:39.444808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4c12b-a576-4abb-9a32-eb1312505a87 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:39.445113 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4c12b-a576-4abb-9a32-eb1312505a87 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:39.445400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4c12b-a576-4abb-9a32-eb1312505a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:39.466386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3ea4c12b-a576-4abb-9a32-eb1312505a87 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:39.467814 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3ea4c12b-a576-4abb-9a32-eb1312505a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:39.503481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098b9b8-6782-49cd-86ac-68c2e3c891f0 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:39.560077 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662f449-5924-4c24-87f7-3ce84804a518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:39.561083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662f449-5924-4c24-87f7-3ce84804a518 None admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:39.561374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662f449-5924-4c24-87f7-3ce84804a518 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:39.561656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662f449-5924-4c24-87f7-3ce84804a518 None admin] PATH_INFO: `/v3/domains/default/users/fcb4950bc93048a891412be785f132b7/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:39.563495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 21/43] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:39.568675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662f449-5924-4c24-87f7-3ce84804a518 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fcb4950bc93048a891412be785f132b7, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:39.568675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662f449-5924-4c24-87f7-3ce84804a518 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:39.583539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 23/44] 10.222.0.52 () {58 vars in 1361 bytes} [Mon Jul 25 08:31:39 2022] PUT /identity/v3/domains/default/users/fcb4950bc93048a891412be785f132b7/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:31:39.593180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4c12b-a576-4abb-9a32-eb1312505a87 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:39.701845 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:39.709965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4c12b-a576-4abb-9a32-eb1312505a87 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:39.749144 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7083e58-e05b-46b4-86cf-8ea19df0b28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:39.749965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7083e58-e05b-46b4-86cf-8ea19df0b28e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:39.750256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7083e58-e05b-46b4-86cf-8ea19df0b28e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:39.750540 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7083e58-e05b-46b4-86cf-8ea19df0b28e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:39.751754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7083e58-e05b-46b4-86cf-8ea19df0b28e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:39.753269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7083e58-e05b-46b4-86cf-8ea19df0b28e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:39.755142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 22/45] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:31:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:39.761206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662f449-5924-4c24-87f7-3ce84804a518 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:39.772935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:39.864223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662f449-5924-4c24-87f7-3ce84804a518 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:39.897976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612656c-f67e-455a-abd7-6b092ee20958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:39.898648 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612656c-f67e-455a-abd7-6b092ee20958 None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:39.898951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612656c-f67e-455a-abd7-6b092ee20958 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:39.899763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612656c-f67e-455a-abd7-6b092ee20958 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:39.900487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612656c-f67e-455a-abd7-6b092ee20958 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:39.901974 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612656c-f67e-455a-abd7-6b092ee20958 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:39.929392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 24/46] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:31:39 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:31:41.588466 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173f944-4f5e-41c9-b9c7-d4c3c4eeb161 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:41.588968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173f944-4f5e-41c9-b9c7-d4c3c4eeb161 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:41.588968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173f944-4f5e-41c9-b9c7-d4c3c4eeb161 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:41.590372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 23/47] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:41.595532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4626be-092a-431b-abba-304307769400 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:41.595956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4626be-092a-431b-abba-304307769400 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:41.596106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4626be-092a-431b-abba-304307769400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:41.614480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4b4626be-092a-431b-abba-304307769400 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:41.616011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4b4626be-092a-431b-abba-304307769400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:41.671666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:41.684593 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 25/48] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:41.707200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173f944-4f5e-41c9-b9c7-d4c3c4eeb161 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:41.807040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173f944-4f5e-41c9-b9c7-d4c3c4eeb161 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:41.839268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a47de-d4e6-4098-97c9-71fb3767ff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:41.840000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a47de-d4e6-4098-97c9-71fb3767ff97 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:41.840289 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a47de-d4e6-4098-97c9-71fb3767ff97 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:41.840597 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a47de-d4e6-4098-97c9-71fb3767ff97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:41.850490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a47de-d4e6-4098-97c9-71fb3767ff97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-d78a47de-d4e6-4098-97c9-71fb3767ff97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:41.858657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:41.859472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:31:41.860284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 24/49] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:31:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:41.865624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4626be-092a-431b-abba-304307769400 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:41.929352 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9fe32-dad2-4a0c-a45a-5f350ad6a047 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:41.929646 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9fe32-dad2-4a0c-a45a-5f350ad6a047 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:41.929927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9fe32-dad2-4a0c-a45a-5f350ad6a047 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:41.931902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 25/50] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:31:41.936731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ec666-940d-44b4-9262-74f2fc03dc97 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:41.937026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ec666-940d-44b4-9262-74f2fc03dc97 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:41.937311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ec666-940d-44b4-9262-74f2fc03dc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:41.954865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-2f2ec666-940d-44b4-9262-74f2fc03dc97 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:41.956051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2f2ec666-940d-44b4-9262-74f2fc03dc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:41.965163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4626be-092a-431b-abba-304307769400 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:41.972286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:42.000169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94323f-8114-4fc2-9abb-e66f5f1a2c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:42.000876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94323f-8114-4fc2-9abb-e66f5f1a2c95 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:42.001165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94323f-8114-4fc2-9abb-e66f5f1a2c95 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:42.001443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94323f-8114-4fc2-9abb-e66f5f1a2c95 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:42.002205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94323f-8114-4fc2-9abb-e66f5f1a2c95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:42.002517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94323f-8114-4fc2-9abb-e66f5f1a2c95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:42.003913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94323f-8114-4fc2-9abb-e66f5f1a2c95 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:42.012294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 26/51] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:42.017915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94323f-8114-4fc2-9abb-e66f5f1a2c95 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:42.025239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 26/52] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:31:42.047812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ec666-940d-44b4-9262-74f2fc03dc97 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:42.072512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:42.120525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94323f-8114-4fc2-9abb-e66f5f1a2c95 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:42.144928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ec666-940d-44b4-9262-74f2fc03dc97 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:42.154335 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b975a4d-1f5e-4012-a7ad-753a917b9790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:42.155222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b975a4d-1f5e-4012-a7ad-753a917b9790 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:42.155510 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b975a4d-1f5e-4012-a7ad-753a917b9790 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:42.155789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b975a4d-1f5e-4012-a7ad-753a917b9790 None admin] PATH_INFO: `/v3/users/fcb4950bc93048a891412be785f132b7` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:42.157272 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b975a4d-1f5e-4012-a7ad-753a917b9790 None admin] RBAC: Authorizing `identity:get_user(user_id=fcb4950bc93048a891412be785f132b7)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:42.158730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b975a4d-1f5e-4012-a7ad-753a917b9790 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:42.160368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 27/53] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:42 2022] GET /identity/v3/users/fcb4950bc93048a891412be785f132b7 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:31:42.166321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b975a4d-1f5e-4012-a7ad-753a917b9790 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:42.179492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea23b80-50e6-4b47-ae9a-038b7598b79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:42.180304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23b80-50e6-4b47-ae9a-038b7598b79d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:42.180617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23b80-50e6-4b47-ae9a-038b7598b79d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:42.180907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23b80-50e6-4b47-ae9a-038b7598b79d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:42.182089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea23b80-50e6-4b47-ae9a-038b7598b79d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:42.183576 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea23b80-50e6-4b47-ae9a-038b7598b79d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:42.185212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 27/54] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:31:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:31:42.190576 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea23b80-50e6-4b47-ae9a-038b7598b79d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:42.212064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:42.264035 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b975a4d-1f5e-4012-a7ad-753a917b9790 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:42.285457 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:42.290597 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea23b80-50e6-4b47-ae9a-038b7598b79d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:42.302175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512464f-b268-4202-9c41-048f3591aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:42.303273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512464f-b268-4202-9c41-048f3591aa64 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:42.303567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512464f-b268-4202-9c41-048f3591aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:42.303848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512464f-b268-4202-9c41-048f3591aa64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:42.305289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512464f-b268-4202-9c41-048f3591aa64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:42.306739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512464f-b268-4202-9c41-048f3591aa64 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:42.308374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 28/55] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:31:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:31:42.313911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512464f-b268-4202-9c41-048f3591aa64 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:42.327543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c950fd8-fb3d-432c-9ce7-7feec18f8602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:42.328211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c950fd8-fb3d-432c-9ce7-7feec18f8602 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:42.328501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c950fd8-fb3d-432c-9ce7-7feec18f8602 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:42.328813 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c950fd8-fb3d-432c-9ce7-7feec18f8602 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:42.329533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c950fd8-fb3d-432c-9ce7-7feec18f8602 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:42.330969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c950fd8-fb3d-432c-9ce7-7feec18f8602 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:42.359004 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 28/56] 10.222.0.52 () {60 vars in 1140 bytes} [Mon Jul 25 08:31:42 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:31:42.424822 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512464f-b268-4202-9c41-048f3591aa64 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:42.461158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc104a-7259-40b8-99f1-fd05fdf16394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:42.461841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc104a-7259-40b8-99f1-fd05fdf16394 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:42.462137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc104a-7259-40b8-99f1-fd05fdf16394 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:42.462417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc104a-7259-40b8-99f1-fd05fdf16394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:42.463268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc104a-7259-40b8-99f1-fd05fdf16394 None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.domain.id=default, user.id=fcb4950bc93048a891412be785f132b7) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:42.463581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc104a-7259-40b8-99f1-fd05fdf16394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:42.465108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc104a-7259-40b8-99f1-fd05fdf16394 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:42.473993 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 29/57] 10.222.0.52 () {56 vars in 1322 bytes} [Mon Jul 25 08:31:42 2022] GET /identity/v3/role_assignments?user.id=fcb4950bc93048a891412be785f132b7&scope.domain.id=default&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:31:44.365449 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3323fb-89bc-4f1d-92d8-8d7a952844ae None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:44.365911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3323fb-89bc-4f1d-92d8-8d7a952844ae None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:44.366027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3323fb-89bc-4f1d-92d8-8d7a952844ae None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:44.367428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 29/58] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:44.372629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba18e68-bb43-4351-ad61-eb661a5f0f8f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:44.373023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba18e68-bb43-4351-ad61-eb661a5f0f8f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:44.373203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba18e68-bb43-4351-ad61-eb661a5f0f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:44.391818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-eba18e68-bb43-4351-ad61-eb661a5f0f8f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:44.393048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-eba18e68-bb43-4351-ad61-eb661a5f0f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:44.439447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:44.459436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 30/59] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:44.482453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3323fb-89bc-4f1d-92d8-8d7a952844ae None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:44.552143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:44.583080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3323fb-89bc-4f1d-92d8-8d7a952844ae None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:44.615121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cca46-c4b8-4fdd-bb46-b5daa91ede4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:44.615907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cca46-c4b8-4fdd-bb46-b5daa91ede4b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:44.616199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cca46-c4b8-4fdd-bb46-b5daa91ede4b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:44.616481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cca46-c4b8-4fdd-bb46-b5daa91ede4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:44.618216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132cca46-c4b8-4fdd-bb46-b5daa91ede4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:44.619680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132cca46-c4b8-4fdd-bb46-b5daa91ede4b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:44.621320 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 30/60] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:31:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:44.626663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba18e68-bb43-4351-ad61-eb661a5f0f8f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:44.723753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba18e68-bb43-4351-ad61-eb661a5f0f8f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:44.756323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2e623-8b83-49f5-bc8f-dfcea7fe154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:44.757019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2e623-8b83-49f5-bc8f-dfcea7fe154b None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:44.757307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2e623-8b83-49f5-bc8f-dfcea7fe154b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:44.757588 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2e623-8b83-49f5-bc8f-dfcea7fe154b None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:44.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2e623-8b83-49f5-bc8f-dfcea7fe154b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:44.760331 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2e623-8b83-49f5-bc8f-dfcea7fe154b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:44.785745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 31/61] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:31:44 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:31:47.778369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702331cf-cd48-4cce-809d-b12205a2f8ec None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:47.778885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702331cf-cd48-4cce-809d-b12205a2f8ec None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:47.779214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702331cf-cd48-4cce-809d-b12205a2f8ec None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:47.780759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 31/62] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:47.795925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99fcee-d7b9-4a36-97a0-1944aefcc16a None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:47.796369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99fcee-d7b9-4a36-97a0-1944aefcc16a None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:47.796602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99fcee-d7b9-4a36-97a0-1944aefcc16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:47.820559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7a99fcee-d7b9-4a36-97a0-1944aefcc16a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:47.821769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7a99fcee-d7b9-4a36-97a0-1944aefcc16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:47.843836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9b8aa-5778-4936-ac09-7bb9537ecd39 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:47.844222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9b8aa-5778-4936-ac09-7bb9537ecd39 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:47.844429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9b8aa-5778-4936-ac09-7bb9537ecd39 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:47.848664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 32/63] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:47.862493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:47.865333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97383e-442e-4e00-8859-ac024c20b01a None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:47.865630 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97383e-442e-4e00-8859-ac024c20b01a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:47.866109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97383e-442e-4e00-8859-ac024c20b01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:47.897570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-2e97383e-442e-4e00-8859-ac024c20b01a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:47.898884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2e97383e-442e-4e00-8859-ac024c20b01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:47.906378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 32/64] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:31:47.928926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99fcee-d7b9-4a36-97a0-1944aefcc16a None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:48.027673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 33/65] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:31:48.040637 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ec704-2b74-44d5-8e07-95b589b5fdd2 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.041135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ec704-2b74-44d5-8e07-95b589b5fdd2 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.041560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ec704-2b74-44d5-8e07-95b589b5fdd2 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.043073 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 34/66] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:31:48.058440 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140daac6-2602-43e1-a5d0-610ae48bda37 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.058942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140daac6-2602-43e1-a5d0-610ae48bda37 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.059389 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140daac6-2602-43e1-a5d0-610ae48bda37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.078740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-140daac6-2602-43e1-a5d0-610ae48bda37 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:48.080136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-140daac6-2602-43e1-a5d0-610ae48bda37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:48.087039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99fcee-d7b9-4a36-97a0-1944aefcc16a None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:48.124728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e492c5-63e3-4952-b958-cc43bf7b57a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:48.125539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e492c5-63e3-4952-b958-cc43bf7b57a3 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.125823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e492c5-63e3-4952-b958-cc43bf7b57a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.126100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e492c5-63e3-4952-b958-cc43bf7b57a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.138467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e492c5-63e3-4952-b958-cc43bf7b57a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-f5e492c5-63e3-4952-b958-cc43bf7b57a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:48.149243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:48.150271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:31:48.151314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 33/67] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:31:48.156413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:48.157585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e492c5-63e3-4952-b958-cc43bf7b57a3 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:48.162223 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 35/68] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:31:48.170259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140daac6-2602-43e1-a5d0-610ae48bda37 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:48.202503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:48.282250 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e492c5-63e3-4952-b958-cc43bf7b57a3 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:48.282876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140daac6-2602-43e1-a5d0-610ae48bda37 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:48.318951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666abcc9-1294-4e16-9858-02a00b802f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:48.319869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666abcc9-1294-4e16-9858-02a00b802f77 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.320158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666abcc9-1294-4e16-9858-02a00b802f77 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.320598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666abcc9-1294-4e16-9858-02a00b802f77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.328627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b8ad0-144b-4c7f-94d1-abd473ad91b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:48.329315 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b8ad0-144b-4c7f-94d1-abd473ad91b3 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.329602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b8ad0-144b-4c7f-94d1-abd473ad91b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.329885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b8ad0-144b-4c7f-94d1-abd473ad91b3 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.330650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b8ad0-144b-4c7f-94d1-abd473ad91b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:48.330956 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b8ad0-144b-4c7f-94d1-abd473ad91b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:48.331323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666abcc9-1294-4e16-9858-02a00b802f77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:48.332348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b8ad0-144b-4c7f-94d1-abd473ad91b3 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-666abcc9-1294-4e16-9858-02a00b802f77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:48.340429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:48.341326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:48.342089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:48.342089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:48.342089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:48.342089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:48.342089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:48.342089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:48.342089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:48.342089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:48.342089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:48.342089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:48.342089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:31:48.342089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:31:48.342651 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 36/69] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:31:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:48.342651 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b8ad0-144b-4c7f-94d1-abd473ad91b3 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:48.350871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 34/70] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:31:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:48.352910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666abcc9-1294-4e16-9858-02a00b802f77 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:48.406651 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:48.448799 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b8ad0-144b-4c7f-94d1-abd473ad91b3 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:48.450261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:48.463830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666abcc9-1294-4e16-9858-02a00b802f77 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:48.487796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13789f-6093-4917-baae-4cf116cf2a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:48.488624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13789f-6093-4917-baae-4cf116cf2a40 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.488913 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13789f-6093-4917-baae-4cf116cf2a40 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.489192 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13789f-6093-4917-baae-4cf116cf2a40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.498012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c674a63-2be0-460a-9c14-bc53f9899a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:48.498890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c674a63-2be0-460a-9c14-bc53f9899a0e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.499176 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c674a63-2be0-460a-9c14-bc53f9899a0e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.499451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c674a63-2be0-460a-9c14-bc53f9899a0e None admin] PATH_INFO: `/v3/users/9dcd945ca11c4739972e841baad81073` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.499608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13789f-6093-4917-baae-4cf116cf2a40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-2e13789f-6093-4917-baae-4cf116cf2a40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:48.507572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:48.509207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 37/71] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:31:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:48.509991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13789f-6093-4917-baae-4cf116cf2a40 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:48.521433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c674a63-2be0-460a-9c14-bc53f9899a0e None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd945ca11c4739972e841baad81073)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:48.523420 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c674a63-2be0-460a-9c14-bc53f9899a0e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:48.526485 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 35/72] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:48 2022] GET /identity/v3/users/9dcd945ca11c4739972e841baad81073 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:48.531475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a73224-0693-478c-a1c6-9d8531f4a62f None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.531475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a73224-0693-478c-a1c6-9d8531f4a62f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.531475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a73224-0693-478c-a1c6-9d8531f4a62f None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.531475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 36/73] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:48.532681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a73224-0693-478c-a1c6-9d8531f4a62f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:48.614292 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13789f-6093-4917-baae-4cf116cf2a40 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:48.636035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:48.644925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a73224-0693-478c-a1c6-9d8531f4a62f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:48.649192 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa28a3-b610-451e-b8da-8fb4c0861323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:48.649876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa28a3-b610-451e-b8da-8fb4c0861323 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.650215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa28a3-b610-451e-b8da-8fb4c0861323 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.650501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa28a3-b610-451e-b8da-8fb4c0861323 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.651274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa28a3-b610-451e-b8da-8fb4c0861323 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:48.651590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa28a3-b610-451e-b8da-8fb4c0861323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:48.653012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa28a3-b610-451e-b8da-8fb4c0861323 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:48.661507 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 38/74] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:31:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:48.662327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa28a3-b610-451e-b8da-8fb4c0861323 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:48.685701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee99ee9-fc6f-4ab9-b93b-a1832b5ff1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:48.686559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee99ee9-fc6f-4ab9-b93b-a1832b5ff1e2 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.686853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee99ee9-fc6f-4ab9-b93b-a1832b5ff1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.687138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee99ee9-fc6f-4ab9-b93b-a1832b5ff1e2 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.687944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee99ee9-fc6f-4ab9-b93b-a1832b5ff1e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:48.688253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee99ee9-fc6f-4ab9-b93b-a1832b5ff1e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:48.689697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee99ee9-fc6f-4ab9-b93b-a1832b5ff1e2 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:48.698383 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 37/75] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:48.699596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea141e2d-02f8-4e33-9f6d-801a399a6eff None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.699888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea141e2d-02f8-4e33-9f6d-801a399a6eff None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.700180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea141e2d-02f8-4e33-9f6d-801a399a6eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.718260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-ea141e2d-02f8-4e33-9f6d-801a399a6eff None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:48.719474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ea141e2d-02f8-4e33-9f6d-801a399a6eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:48.721486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:48.762357 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa28a3-b610-451e-b8da-8fb4c0861323 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:48.787788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 38/76] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:48.789154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffa546-6ea9-49f4-bd11-ee04ad6f4f08 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.789452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffa546-6ea9-49f4-bd11-ee04ad6f4f08 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.789746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffa546-6ea9-49f4-bd11-ee04ad6f4f08 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.791151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 39/77] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:48.791703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffa546-6ea9-49f4-bd11-ee04ad6f4f08 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:48.801661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6f6d7-2685-4e58-8de1-903f6f64a9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:48.802822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6f6d7-2685-4e58-8de1-903f6f64a9d9 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.803111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6f6d7-2685-4e58-8de1-903f6f64a9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.803390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6f6d7-2685-4e58-8de1-903f6f64a9d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.854969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:48.870972 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6f6d7-2685-4e58-8de1-903f6f64a9d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:48.872379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6f6d7-2685-4e58-8de1-903f6f64a9d9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:48.899328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffa546-6ea9-49f4-bd11-ee04ad6f4f08 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:48.932914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6698e-4918-4889-a254-6ec715c8316b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:48.933782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6698e-4918-4889-a254-6ec715c8316b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.934066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6698e-4918-4889-a254-6ec715c8316b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.934341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6698e-4918-4889-a254-6ec715c8316b None admin] PATH_INFO: `/v3/users/9dcd945ca11c4739972e841baad81073` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.935547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6698e-4918-4889-a254-6ec715c8316b None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd945ca11c4739972e841baad81073)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:48.937035 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6698e-4918-4889-a254-6ec715c8316b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:48.938690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 40/78] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:48 2022] GET /identity/v3/users/9dcd945ca11c4739972e841baad81073 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:48.939453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6698e-4918-4889-a254-6ec715c8316b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:48.948087 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:48.974039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 39/79] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:31:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:31:48.975318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536c1c3-a5d4-429d-8b22-7817c6df4121 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:48.975612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536c1c3-a5d4-429d-8b22-7817c6df4121 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:48.975896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536c1c3-a5d4-429d-8b22-7817c6df4121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:48.993180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-e536c1c3-a5d4-429d-8b22-7817c6df4121 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:48.994378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e536c1c3-a5d4-429d-8b22-7817c6df4121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:49.039234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6698e-4918-4889-a254-6ec715c8316b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:49.058943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 40/80] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:49.059765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536c1c3-a5d4-429d-8b22-7817c6df4121 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:49.072591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c4151-101a-4542-bc72-52831a8b6ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:49.073484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c4151-101a-4542-bc72-52831a8b6ae8 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:49.073771 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c4151-101a-4542-bc72-52831a8b6ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:49.074045 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c4151-101a-4542-bc72-52831a8b6ae8 None admin] PATH_INFO: `/v3/users/fcb4950bc93048a891412be785f132b7` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:49.075240 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c4151-101a-4542-bc72-52831a8b6ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=fcb4950bc93048a891412be785f132b7)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:49.076699 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c4151-101a-4542-bc72-52831a8b6ae8 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:49.078316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 41/81] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:48 2022] GET /identity/v3/users/fcb4950bc93048a891412be785f132b7 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:49.079183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c4151-101a-4542-bc72-52831a8b6ae8 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:49.090643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:49.155435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:49.174416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536c1c3-a5d4-429d-8b22-7817c6df4121 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:49.180560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c4151-101a-4542-bc72-52831a8b6ae8 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:49.207980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa11801-f2d0-434e-9dcb-f5f082df8373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:49.208816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa11801-f2d0-434e-9dcb-f5f082df8373 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:49.209119 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa11801-f2d0-434e-9dcb-f5f082df8373 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:49.209402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa11801-f2d0-434e-9dcb-f5f082df8373 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:49.213686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcd069-c46d-4e49-9355-05552ae308af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:49.214352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcd069-c46d-4e49-9355-05552ae308af None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:49.214620 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcd069-c46d-4e49-9355-05552ae308af None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:49.214899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcd069-c46d-4e49-9355-05552ae308af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:49.219545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa11801-f2d0-434e-9dcb-f5f082df8373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-7aa11801-f2d0-434e-9dcb-f5f082df8373 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:49.227355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:49.228333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:31:49.229376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 41/82] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:49.231361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa11801-f2d0-434e-9dcb-f5f082df8373 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:49.281801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcd069-c46d-4e49-9355-05552ae308af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:49.283312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcd069-c46d-4e49-9355-05552ae308af None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:49.308847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:49.333176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa11801-f2d0-434e-9dcb-f5f082df8373 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:49.358967 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:49.366230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ab7a5-1ba3-457b-857f-c8fa8a864178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:49.367102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ab7a5-1ba3-457b-857f-c8fa8a864178 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:49.367974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ab7a5-1ba3-457b-857f-c8fa8a864178 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:49.368252 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ab7a5-1ba3-457b-857f-c8fa8a864178 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:49.369467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ab7a5-1ba3-457b-857f-c8fa8a864178 None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:49.370917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ab7a5-1ba3-457b-857f-c8fa8a864178 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:49.372635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 42/83] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:31:49 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:49.373395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ab7a5-1ba3-457b-857f-c8fa8a864178 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:49.385294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 42/84] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:31:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:31:49.386482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcd069-c46d-4e49-9355-05552ae308af None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:49.474872 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ab7a5-1ba3-457b-857f-c8fa8a864178 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:49.485141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcd069-c46d-4e49-9355-05552ae308af None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:49.508761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f9a89-879e-45a9-9da4-fa1c1b1ce545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:49.509558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f9a89-879e-45a9-9da4-fa1c1b1ce545 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:49.509845 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f9a89-879e-45a9-9da4-fa1c1b1ce545 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:49.510124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f9a89-879e-45a9-9da4-fa1c1b1ce545 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:49.511303 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f9a89-879e-45a9-9da4-fa1c1b1ce545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:49.512789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f9a89-879e-45a9-9da4-fa1c1b1ce545 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:49.518845 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 43/85] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:31:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:49.518845 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f9a89-879e-45a9-9da4-fa1c1b1ce545 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:49.520308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de7db9-2a7c-4523-bc1f-a27e2147146a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:49.521072 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de7db9-2a7c-4523-bc1f-a27e2147146a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:49.521370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de7db9-2a7c-4523-bc1f-a27e2147146a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:49.521655 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de7db9-2a7c-4523-bc1f-a27e2147146a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:49.526888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:49.565689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:49.591126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de7db9-2a7c-4523-bc1f-a27e2147146a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:49.592660 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de7db9-2a7c-4523-bc1f-a27e2147146a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:49.613943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f9a89-879e-45a9-9da4-fa1c1b1ce545 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:49.646924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86d4ab-0170-48d5-99b1-dac5a5698447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:49.647594 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86d4ab-0170-48d5-99b1-dac5a5698447 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:49.647883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86d4ab-0170-48d5-99b1-dac5a5698447 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:49.648169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86d4ab-0170-48d5-99b1-dac5a5698447 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:49.649467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86d4ab-0170-48d5-99b1-dac5a5698447 None admin] RBAC: Adding query filter params (name=member) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:49.649778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86d4ab-0170-48d5-99b1-dac5a5698447 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:49.651134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86d4ab-0170-48d5-99b1-dac5a5698447 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:49.659502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 44/86] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:31:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:49.660157 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86d4ab-0170-48d5-99b1-dac5a5698447 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:49.694837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 43/87] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:31:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:31:49.695709 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de7db9-2a7c-4523-bc1f-a27e2147146a None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:49.727842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:49.756919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86d4ab-0170-48d5-99b1-dac5a5698447 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:49.769611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:49.792640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f0cf9-4ee3-4322-b350-36f0c76ccb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:49.793322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f0cf9-4ee3-4322-b350-36f0c76ccb0c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:49.793612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f0cf9-4ee3-4322-b350-36f0c76ccb0c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:49.793888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f0cf9-4ee3-4322-b350-36f0c76ccb0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:49.794721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f0cf9-4ee3-4322-b350-36f0c76ccb0c None admin] RBAC: Adding query filter params (role.id=82ee29c384454b9aa4db085c9d2e21e8, scope.project.id=d68d17929cf4421f977087a339234c2a, user.id=9dcd945ca11c4739972e841baad81073) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:49.795035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f0cf9-4ee3-4322-b350-36f0c76ccb0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:49.796477 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f0cf9-4ee3-4322-b350-36f0c76ccb0c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:49.797091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de7db9-2a7c-4523-bc1f-a27e2147146a None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:49.802040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 45/88] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:31:49 2022] GET /identity/v3/role_assignments?user.id=9dcd945ca11c4739972e841baad81073&scope.project.id=d68d17929cf4421f977087a339234c2a&role.id=82ee29c384454b9aa4db085c9d2e21e8 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:49.802653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f0cf9-4ee3-4322-b350-36f0c76ccb0c None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:49.848348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924391ca-2e1f-438b-834f-be1bff25fc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:49.849299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924391ca-2e1f-438b-834f-be1bff25fc2f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:49.849591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924391ca-2e1f-438b-834f-be1bff25fc2f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:49.849881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924391ca-2e1f-438b-834f-be1bff25fc2f None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:49.851217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924391ca-2e1f-438b-834f-be1bff25fc2f None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:49.853446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924391ca-2e1f-438b-834f-be1bff25fc2f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:49.855311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 44/89] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:31:49 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:49.856137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924391ca-2e1f-438b-834f-be1bff25fc2f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:49.920158 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f0cf9-4ee3-4322-b350-36f0c76ccb0c None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:49.958990 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace96bcd-f8f6-4841-94f8-a303512ee2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:49.959691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace96bcd-f8f6-4841-94f8-a303512ee2e8 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:49.959987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace96bcd-f8f6-4841-94f8-a303512ee2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:49.960275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace96bcd-f8f6-4841-94f8-a303512ee2e8 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:49.961577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace96bcd-f8f6-4841-94f8-a303512ee2e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:49.963050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace96bcd-f8f6-4841-94f8-a303512ee2e8 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:49.974847 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924391ca-2e1f-438b-834f-be1bff25fc2f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:49.977388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:50.000701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 46/90] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:31:49 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:31:50.000701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace96bcd-f8f6-4841-94f8-a303512ee2e8 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:50.015227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bac48-e55d-4a8e-b5d8-885da3f40545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:50.016298 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bac48-e55d-4a8e-b5d8-885da3f40545 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:50.016597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bac48-e55d-4a8e-b5d8-885da3f40545 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:50.016889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bac48-e55d-4a8e-b5d8-885da3f40545 None admin] PATH_INFO: `/v3/users/9dcd945ca11c4739972e841baad81073` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:50.018359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0bac48-e55d-4a8e-b5d8-885da3f40545 None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd945ca11c4739972e841baad81073)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:50.020163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0bac48-e55d-4a8e-b5d8-885da3f40545 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:50.023128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 45/91] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:49 2022] GET /identity/v3/users/9dcd945ca11c4739972e841baad81073 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:50.024687 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bac48-e55d-4a8e-b5d8-885da3f40545 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:50.039406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:50.144597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bac48-e55d-4a8e-b5d8-885da3f40545 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:50.166871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace96bcd-f8f6-4841-94f8-a303512ee2e8 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:50.189744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ae7a6-47e5-4ec7-ba8b-12e060954868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:50.190417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ae7a6-47e5-4ec7-ba8b-12e060954868 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:50.190702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ae7a6-47e5-4ec7-ba8b-12e060954868 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:50.190979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ae7a6-47e5-4ec7-ba8b-12e060954868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:50.191819 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ae7a6-47e5-4ec7-ba8b-12e060954868 None admin] RBAC: Adding query filter params (role.id=3ce6c4ec0ed142828cb20882e342328e, scope.project.id=d68d17929cf4421f977087a339234c2a, user.id=9dcd945ca11c4739972e841baad81073) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:50.192133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ae7a6-47e5-4ec7-ba8b-12e060954868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:50.193645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ae7a6-47e5-4ec7-ba8b-12e060954868 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:50.202099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 46/92] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:31:50 2022] GET /identity/v3/role_assignments?user.id=9dcd945ca11c4739972e841baad81073&scope.project.id=d68d17929cf4421f977087a339234c2a&role.id=3ce6c4ec0ed142828cb20882e342328e => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:50.202359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ae7a6-47e5-4ec7-ba8b-12e060954868 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:50.209192 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8040be-dfb1-42f5-b1e1-549b07377702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:50.210238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8040be-dfb1-42f5-b1e1-549b07377702 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:50.210629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8040be-dfb1-42f5-b1e1-549b07377702 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:50.211581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8040be-dfb1-42f5-b1e1-549b07377702 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:50.213146 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8040be-dfb1-42f5-b1e1-549b07377702 None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:50.214646 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8040be-dfb1-42f5-b1e1-549b07377702 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:50.216451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 47/93] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:31:49 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:50.222334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8040be-dfb1-42f5-b1e1-549b07377702 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:50.312986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:50.322891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:50.350937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8040be-dfb1-42f5-b1e1-549b07377702 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:50.363446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ae7a6-47e5-4ec7-ba8b-12e060954868 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:50.402934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa812fd7-07f7-48e3-8a3e-f37860203c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:50.403963 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa812fd7-07f7-48e3-8a3e-f37860203c96 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:50.404449 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa812fd7-07f7-48e3-8a3e-f37860203c96 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:50.405028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa812fd7-07f7-48e3-8a3e-f37860203c96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:50.406276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa812fd7-07f7-48e3-8a3e-f37860203c96 None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=d68d17929cf4421f977087a339234c2a, user.id=fcb4950bc93048a891412be785f132b7) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:50.406737 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa812fd7-07f7-48e3-8a3e-f37860203c96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:50.408622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30efe75-82ee-4315-b7c2-957c19901f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:50.408798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30efe75-82ee-4315-b7c2-957c19901f84 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:50.409204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30efe75-82ee-4315-b7c2-957c19901f84 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:50.409507 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa812fd7-07f7-48e3-8a3e-f37860203c96 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:50.409661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30efe75-82ee-4315-b7c2-957c19901f84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:50.418458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 48/94] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:31:50 2022] GET /identity/v3/role_assignments?user.id=fcb4950bc93048a891412be785f132b7&scope.project.id=d68d17929cf4421f977087a339234c2a&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:50.493632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30efe75-82ee-4315-b7c2-957c19901f84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:50.495166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30efe75-82ee-4315-b7c2-957c19901f84 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:50.589799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:50.621592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 47/95] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:31:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:31:50.626930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa812fd7-07f7-48e3-8a3e-f37860203c96 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:50.768482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa812fd7-07f7-48e3-8a3e-f37860203c96 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:50.825432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:50.853535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d82df-5548-4a5b-a2b6-c4953d2555de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:50.854692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d82df-5548-4a5b-a2b6-c4953d2555de None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:50.855162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d82df-5548-4a5b-a2b6-c4953d2555de None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:50.855577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d82df-5548-4a5b-a2b6-c4953d2555de None admin] PATH_INFO: `/v3/projects/5aaef6077b164f09882021c210f75662` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:50.861304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d82df-5548-4a5b-a2b6-c4953d2555de None admin] RBAC: Authorizing `identity:get_project(project_id=5aaef6077b164f09882021c210f75662)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:50.862931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d82df-5548-4a5b-a2b6-c4953d2555de None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:50.864838 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 49/96] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:31:50 2022] GET /identity/v3/projects/5aaef6077b164f09882021c210f75662 => generated 312 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:50.871372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30efe75-82ee-4315-b7c2-957c19901f84 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:51.027894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30efe75-82ee-4315-b7c2-957c19901f84 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:51.067745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bf075-0432-4693-8823-b77b18277184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:51.068423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bf075-0432-4693-8823-b77b18277184 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:51.068739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bf075-0432-4693-8823-b77b18277184 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:51.069021 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bf075-0432-4693-8823-b77b18277184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:51.069870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bf075-0432-4693-8823-b77b18277184 None admin] RBAC: Adding query filter params (role.id=82ee29c384454b9aa4db085c9d2e21e8, scope.project.id=5aaef6077b164f09882021c210f75662, user.id=9dcd945ca11c4739972e841baad81073) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:51.070722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bf075-0432-4693-8823-b77b18277184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:51.072188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bf075-0432-4693-8823-b77b18277184 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:51.085446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 48/97] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:31:50 2022] GET /identity/v3/role_assignments?user.id=9dcd945ca11c4739972e841baad81073&scope.project.id=5aaef6077b164f09882021c210f75662&role.id=82ee29c384454b9aa4db085c9d2e21e8 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:52.574827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70db6dc-da88-4e1e-a4a7-04b66bf3bfc9 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:52.575304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70db6dc-da88-4e1e-a4a7-04b66bf3bfc9 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:52.575304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70db6dc-da88-4e1e-a4a7-04b66bf3bfc9 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:52.576893 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 50/98] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:52.581980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e78543-ed16-4733-abf1-d9770c21bf7b None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:52.582386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e78543-ed16-4733-abf1-d9770c21bf7b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:52.582550 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e78543-ed16-4733-abf1-d9770c21bf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:52.609569 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-84e78543-ed16-4733-abf1-d9770c21bf7b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:52.610791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-84e78543-ed16-4733-abf1-d9770c21bf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:52.645834 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:52.696770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 49/99] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:52.730892 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70db6dc-da88-4e1e-a4a7-04b66bf3bfc9 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:52.848321 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70db6dc-da88-4e1e-a4a7-04b66bf3bfc9 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:52.892389 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1573e-41ef-48bf-9d56-b320d7f989e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:52.893508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1573e-41ef-48bf-9d56-b320d7f989e1 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:52.893798 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1573e-41ef-48bf-9d56-b320d7f989e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:52.894074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1573e-41ef-48bf-9d56-b320d7f989e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:52.913448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1573e-41ef-48bf-9d56-b320d7f989e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-fef1573e-41ef-48bf-9d56-b320d7f989e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:52.921490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:52.922334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:31:52.923088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 51/100] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:52.930183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e78543-ed16-4733-abf1-d9770c21bf7b None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:53.040676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e78543-ed16-4733-abf1-d9770c21bf7b None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:53.064611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc4cb5-0184-4cdc-9b8e-0d1f61f1cb57 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.064902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc4cb5-0184-4cdc-9b8e-0d1f61f1cb57 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.065146 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc4cb5-0184-4cdc-9b8e-0d1f61f1cb57 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.066529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 52/101] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:31:53.071431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca46d8-6d5c-479c-a62f-baedbfd60ce8 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.071719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca46d8-6d5c-479c-a62f-baedbfd60ce8 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.072000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca46d8-6d5c-479c-a62f-baedbfd60ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.098931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-98ca46d8-6d5c-479c-a62f-baedbfd60ce8 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:53.100054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-98ca46d8-6d5c-479c-a62f-baedbfd60ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:53.103209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d0396-2f2c-4423-9fe3-60fe4dd93d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:53.103896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d0396-2f2c-4423-9fe3-60fe4dd93d4b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.104181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d0396-2f2c-4423-9fe3-60fe4dd93d4b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.104465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d0396-2f2c-4423-9fe3-60fe4dd93d4b None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.105256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d0396-2f2c-4423-9fe3-60fe4dd93d4b None admin] RBAC: Adding query filter params (name=member) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:53.105566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d0396-2f2c-4423-9fe3-60fe4dd93d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:53.107042 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d0396-2f2c-4423-9fe3-60fe4dd93d4b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:53.123347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 50/102] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:31:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:31:53.133551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d0396-2f2c-4423-9fe3-60fe4dd93d4b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:53.200192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:53.228392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:53.283322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 53/103] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:53.303656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1164d0b-32cd-432a-9e1a-7e5da9c76fc5 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.303656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1164d0b-32cd-432a-9e1a-7e5da9c76fc5 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.303656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1164d0b-32cd-432a-9e1a-7e5da9c76fc5 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.305968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 54/104] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:53.312595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a272ee5-5161-4b01-85c3-3b9a883e472d None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.312595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a272ee5-5161-4b01-85c3-3b9a883e472d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.312595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a272ee5-5161-4b01-85c3-3b9a883e472d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.322084 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d0396-2f2c-4423-9fe3-60fe4dd93d4b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:53.329609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3a272ee5-5161-4b01-85c3-3b9a883e472d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:53.330998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3a272ee5-5161-4b01-85c3-3b9a883e472d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:53.371655 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f2207-dc59-4d5e-a1c5-76485a9937ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:53.372474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f2207-dc59-4d5e-a1c5-76485a9937ad None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.372796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f2207-dc59-4d5e-a1c5-76485a9937ad None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.373078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f2207-dc59-4d5e-a1c5-76485a9937ad None admin] PATH_INFO: `/v3/users/9dcd945ca11c4739972e841baad81073` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.375002 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f2207-dc59-4d5e-a1c5-76485a9937ad None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd945ca11c4739972e841baad81073)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:53.376475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f2207-dc59-4d5e-a1c5-76485a9937ad None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:53.378167 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 51/105] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:53 2022] GET /identity/v3/users/9dcd945ca11c4739972e841baad81073 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:53.378945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f2207-dc59-4d5e-a1c5-76485a9937ad None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:53.466287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 55/106] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:31:53.467183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a272ee5-5161-4b01-85c3-3b9a883e472d None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:53.514603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:53.526721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f2207-dc59-4d5e-a1c5-76485a9937ad None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:53.575318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646ecfb-b0e3-43bb-b144-405ee76b11cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:53.576145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646ecfb-b0e3-43bb-b144-405ee76b11cd None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.576434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646ecfb-b0e3-43bb-b144-405ee76b11cd None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.576751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646ecfb-b0e3-43bb-b144-405ee76b11cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.591387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3646ecfb-b0e3-43bb-b144-405ee76b11cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-3646ecfb-b0e3-43bb-b144-405ee76b11cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:53.603695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:53.604537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:53.605380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:53.605380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:53.605380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:53.605380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:53.605380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:53.605380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:53.605380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:53.605380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:53.605380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:53.605380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:53.605380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:31:53.605380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:31:53.606366 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 52/107] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:31:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:53.606432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a272ee5-5161-4b01-85c3-3b9a883e472d None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:53.607052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646ecfb-b0e3-43bb-b144-405ee76b11cd None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:53.622847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:53.649169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ecb08-fbd6-439b-941e-fb38750c8db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:53.650048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ecb08-fbd6-439b-941e-fb38750c8db3 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.650336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ecb08-fbd6-439b-941e-fb38750c8db3 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.650610 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ecb08-fbd6-439b-941e-fb38750c8db3 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.651797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ecb08-fbd6-439b-941e-fb38750c8db3 None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:53.653285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ecb08-fbd6-439b-941e-fb38750c8db3 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:53.655161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 56/108] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:31:53 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:53.656429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f754480-3ac3-4273-8045-89e17b1f890b None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.656757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f754480-3ac3-4273-8045-89e17b1f890b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.657037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f754480-3ac3-4273-8045-89e17b1f890b None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.658395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 57/109] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:53.658970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f754480-3ac3-4273-8045-89e17b1f890b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:53.742925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646ecfb-b0e3-43bb-b144-405ee76b11cd None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:53.762302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f754480-3ac3-4273-8045-89e17b1f890b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:53.778737 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c705b-0bec-45a4-8a72-95d8baabd757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:53.779547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c705b-0bec-45a4-8a72-95d8baabd757 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.779839 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c705b-0bec-45a4-8a72-95d8baabd757 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.780128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c705b-0bec-45a4-8a72-95d8baabd757 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.791115 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c705b-0bec-45a4-8a72-95d8baabd757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:53.797434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922958c-0e9d-4d96-ab7f-de515387c43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:53.798109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922958c-0e9d-4d96-ab7f-de515387c43e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.798395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922958c-0e9d-4d96-ab7f-de515387c43e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.798670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922958c-0e9d-4d96-ab7f-de515387c43e None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-cd8c705b-0bec-45a4-8a72-95d8baabd757 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:53.798889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:53.799697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:53.805824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:53.805824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:53.805824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:53.805824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:53.805824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:53.805824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:53.805824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:53.805824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:53.805824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:53.805824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:53.805824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:31:53.805824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:31:53.806403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922958c-0e9d-4d96-ab7f-de515387c43e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:53.806403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922958c-0e9d-4d96-ab7f-de515387c43e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:53.806520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 53/110] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:31:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:53.806581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922958c-0e9d-4d96-ab7f-de515387c43e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:53.806642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d09a6-9734-4262-b45f-48f5983d905c None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.806642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d09a6-9734-4262-b45f-48f5983d905c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.806642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d09a6-9734-4262-b45f-48f5983d905c None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.806642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 54/111] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:53.806642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d09a6-9734-4262-b45f-48f5983d905c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:53.814783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 58/112] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:31:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:53.816044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6be57-d666-4acd-a491-f6cde62b8612 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.816337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6be57-d666-4acd-a491-f6cde62b8612 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.817764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6be57-d666-4acd-a491-f6cde62b8612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.830573 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:53.837634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-4db6be57-d666-4acd-a491-f6cde62b8612 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:53.838783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4db6be57-d666-4acd-a491-f6cde62b8612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:53.875897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:53.912001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 59/113] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:53.913276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6be57-d666-4acd-a491-f6cde62b8612 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:53.919503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d09a6-9734-4262-b45f-48f5983d905c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:53.955105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3cade9-6586-497c-9dc3-432e76c5001a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:53.956011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3cade9-6586-497c-9dc3-432e76c5001a None admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.956299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3cade9-6586-497c-9dc3-432e76c5001a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.956592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3cade9-6586-497c-9dc3-432e76c5001a None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a/users/9dcd945ca11c4739972e841baad81073/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:53.958579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3cade9-6586-497c-9dc3-432e76c5001a None admin] RBAC: Authorizing `identity:create_grant(project_id=d68d17929cf4421f977087a339234c2a, user_id=9dcd945ca11c4739972e841baad81073, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:53.960062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3cade9-6586-497c-9dc3-432e76c5001a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:53.974082 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 55/114] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:31:53 2022] PUT /identity/v3/projects/d68d17929cf4421f977087a339234c2a/users/9dcd945ca11c4739972e841baad81073/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:31:53.975305 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419fd86-4956-4346-ad0a-ca31b2d5efe3 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:53.975580 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419fd86-4956-4346-ad0a-ca31b2d5efe3 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:53.975863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419fd86-4956-4346-ad0a-ca31b2d5efe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.004916 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-e419fd86-4956-4346-ad0a-ca31b2d5efe3 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:54.006270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e419fd86-4956-4346-ad0a-ca31b2d5efe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:54.027368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6be57-d666-4acd-a491-f6cde62b8612 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:54.066977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0b695-4e1f-48c2-8f38-6a3be6741425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:54.067705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0b695-4e1f-48c2-8f38-6a3be6741425 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:54.067999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0b695-4e1f-48c2-8f38-6a3be6741425 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:54.068285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0b695-4e1f-48c2-8f38-6a3be6741425 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.069167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0b695-4e1f-48c2-8f38-6a3be6741425 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:54.069486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0b695-4e1f-48c2-8f38-6a3be6741425 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:54.072010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0b695-4e1f-48c2-8f38-6a3be6741425 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:54.081784 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 60/115] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:54.082509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0b695-4e1f-48c2-8f38-6a3be6741425 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:54.096611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 56/116] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:54.096611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2bea6-9e0e-43d2-abe7-89060b45d938 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:54.096611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2bea6-9e0e-43d2-abe7-89060b45d938 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:54.096611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2bea6-9e0e-43d2-abe7-89060b45d938 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.096611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 57/117] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:54.096611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2bea6-9e0e-43d2-abe7-89060b45d938 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:54.110889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:54.135409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:54.209927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0b695-4e1f-48c2-8f38-6a3be6741425 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:54.212385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2bea6-9e0e-43d2-abe7-89060b45d938 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:54.246250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647ab11-f622-43a4-b3c3-71bcc4f05588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:54.247229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02e399-d8db-43a4-8476-987b2024ded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:54.247608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647ab11-f622-43a4-b3c3-71bcc4f05588 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:54.248037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02e399-d8db-43a4-8476-987b2024ded1 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:54.248263 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647ab11-f622-43a4-b3c3-71bcc4f05588 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:54.248357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02e399-d8db-43a4-8476-987b2024ded1 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:54.248628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02e399-d8db-43a4-8476-987b2024ded1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.248992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647ab11-f622-43a4-b3c3-71bcc4f05588 None admin] PATH_INFO: `/v3/users/9dcd945ca11c4739972e841baad81073` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.249837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02e399-d8db-43a4-8476-987b2024ded1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:54.250531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647ab11-f622-43a4-b3c3-71bcc4f05588 None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd945ca11c4739972e841baad81073)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:54.251291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02e399-d8db-43a4-8476-987b2024ded1 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:54.252324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647ab11-f622-43a4-b3c3-71bcc4f05588 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:54.252974 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 58/118] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:31:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:54.253747 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02e399-d8db-43a4-8476-987b2024ded1 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:54.254596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 61/119] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:54 2022] GET /identity/v3/users/9dcd945ca11c4739972e841baad81073 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:54.255951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb665bd-0cc3-4a2a-b262-34ee4b62f3b5 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:54.256397 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb665bd-0cc3-4a2a-b262-34ee4b62f3b5 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:54.256855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb665bd-0cc3-4a2a-b262-34ee4b62f3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.276743 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-bcb665bd-0cc3-4a2a-b262-34ee4b62f3b5 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:54.278373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-bcb665bd-0cc3-4a2a-b262-34ee4b62f3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:54.346203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:54.354000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 62/120] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:54.354672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb665bd-0cc3-4a2a-b262-34ee4b62f3b5 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:54.384877 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02e399-d8db-43a4-8476-987b2024ded1 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:54.386996 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:54.423088 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe4f36-2c0b-47fc-b21c-05a5e7425a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:54.423984 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe4f36-2c0b-47fc-b21c-05a5e7425a13 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:54.424272 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe4f36-2c0b-47fc-b21c-05a5e7425a13 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:54.424551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe4f36-2c0b-47fc-b21c-05a5e7425a13 None admin] PATH_INFO: `/v3/users/fcb4950bc93048a891412be785f132b7` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.425798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe4f36-2c0b-47fc-b21c-05a5e7425a13 None admin] RBAC: Authorizing `identity:get_user(user_id=fcb4950bc93048a891412be785f132b7)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:54.427269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe4f36-2c0b-47fc-b21c-05a5e7425a13 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:54.433854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 59/121] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:54 2022] GET /identity/v3/users/fcb4950bc93048a891412be785f132b7 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:54.433854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe4f36-2c0b-47fc-b21c-05a5e7425a13 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:54.464699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb665bd-0cc3-4a2a-b262-34ee4b62f3b5 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:54.500260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16e968-189d-4150-b94a-8f93e52eab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:54.501096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16e968-189d-4150-b94a-8f93e52eab12 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:54.501382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16e968-189d-4150-b94a-8f93e52eab12 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:54.501660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16e968-189d-4150-b94a-8f93e52eab12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.512560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16e968-189d-4150-b94a-8f93e52eab12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-3e16e968-189d-4150-b94a-8f93e52eab12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:54.520399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:54.521446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:31:54.522284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 63/122] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:54.522899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16e968-189d-4150-b94a-8f93e52eab12 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:54.545270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe4f36-2c0b-47fc-b21c-05a5e7425a13 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:54.579706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff48aa-da0a-45b9-94df-de8a02e083a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:54.580383 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff48aa-da0a-45b9-94df-de8a02e083a6 None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:54.580703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff48aa-da0a-45b9-94df-de8a02e083a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:54.581271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff48aa-da0a-45b9-94df-de8a02e083a6 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.581712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff48aa-da0a-45b9-94df-de8a02e083a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:54.583183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff48aa-da0a-45b9-94df-de8a02e083a6 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:54.599857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:54.613969 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 60/123] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:31:54 2022] POST /identity/v3/projects => generated 302 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:31:54.615103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff48aa-da0a-45b9-94df-de8a02e083a6 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:54.640753 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:54.656628 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16e968-189d-4150-b94a-8f93e52eab12 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:54.709203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4353a-7c61-42a1-bc6f-185b9e17ffec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:54.711413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4353a-7c61-42a1-bc6f-185b9e17ffec None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:54.711926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4353a-7c61-42a1-bc6f-185b9e17ffec None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:54.712456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4353a-7c61-42a1-bc6f-185b9e17ffec None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.714244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4353a-7c61-42a1-bc6f-185b9e17ffec None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:54.715977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4353a-7c61-42a1-bc6f-185b9e17ffec None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:54.718296 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 64/124] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:31:54 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:54.724224 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4353a-7c61-42a1-bc6f-185b9e17ffec None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:54.735098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff48aa-da0a-45b9-94df-de8a02e083a6 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:54.773899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c1983-e926-45a3-9854-2fad7bc3359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:54.774431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c1983-e926-45a3-9854-2fad7bc3359f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:54.774717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c1983-e926-45a3-9854-2fad7bc3359f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:54.775807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c1983-e926-45a3-9854-2fad7bc3359f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.787378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c1983-e926-45a3-9854-2fad7bc3359f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-d43c1983-e926-45a3-9854-2fad7bc3359f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:54.795740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:54.796642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:31:54.797422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 61/125] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:54.798071 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c1983-e926-45a3-9854-2fad7bc3359f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:54.857902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4353a-7c61-42a1-bc6f-185b9e17ffec None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:54.870008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:54.894668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735900c-82dd-4a8a-a183-2fb9c8849ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:54.895575 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735900c-82dd-4a8a-a183-2fb9c8849ed0 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:54.895931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735900c-82dd-4a8a-a183-2fb9c8849ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:54.896563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735900c-82dd-4a8a-a183-2fb9c8849ed0 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.897617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735900c-82dd-4a8a-a183-2fb9c8849ed0 None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:54.899717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735900c-82dd-4a8a-a183-2fb9c8849ed0 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:54.901869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 65/126] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:31:54 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:54.902510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735900c-82dd-4a8a-a183-2fb9c8849ed0 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:54.930147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c1983-e926-45a3-9854-2fad7bc3359f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:54.956977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:54.975557 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728984c-6848-4a74-beb3-120d29317982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:54.977050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728984c-6848-4a74-beb3-120d29317982 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:54.977515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728984c-6848-4a74-beb3-120d29317982 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:54.977995 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728984c-6848-4a74-beb3-120d29317982 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:54.978919 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728984c-6848-4a74-beb3-120d29317982 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:54.979397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728984c-6848-4a74-beb3-120d29317982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:54.981378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728984c-6848-4a74-beb3-120d29317982 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:54.993083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 62/127] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:31:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:54.994278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728984c-6848-4a74-beb3-120d29317982 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:55.011752 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735900c-82dd-4a8a-a183-2fb9c8849ed0 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:55.052835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42233c-727f-479c-9895-b336c6384bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:55.053690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42233c-727f-479c-9895-b336c6384bb9 None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:55.053984 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42233c-727f-479c-9895-b336c6384bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:55.054306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42233c-727f-479c-9895-b336c6384bb9 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a/users/9dcd945ca11c4739972e841baad81073/roles/3ce6c4ec0ed142828cb20882e342328e` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:55.056850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42233c-727f-479c-9895-b336c6384bb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d68d17929cf4421f977087a339234c2a, user_id=9dcd945ca11c4739972e841baad81073, role_id=3ce6c4ec0ed142828cb20882e342328e)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:55.058325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42233c-727f-479c-9895-b336c6384bb9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:55.080804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 66/128] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:31:54 2022] PUT /identity/v3/projects/d68d17929cf4421f977087a339234c2a/users/9dcd945ca11c4739972e841baad81073/roles/3ce6c4ec0ed142828cb20882e342328e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:31:55.085083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42233c-727f-479c-9895-b336c6384bb9 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:55.155961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728984c-6848-4a74-beb3-120d29317982 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:55.191038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:55.226849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824fefe-856a-4259-a5c5-235bb90f3f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:55.227545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824fefe-856a-4259-a5c5-235bb90f3f86 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:55.227830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824fefe-856a-4259-a5c5-235bb90f3f86 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:55.228108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824fefe-856a-4259-a5c5-235bb90f3f86 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:55.228903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824fefe-856a-4259-a5c5-235bb90f3f86 None admin] RBAC: Adding query filter params (name=member) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:55.229214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824fefe-856a-4259-a5c5-235bb90f3f86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:55.230629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824fefe-856a-4259-a5c5-235bb90f3f86 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:55.246953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 63/129] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:31:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:55.247867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824fefe-856a-4259-a5c5-235bb90f3f86 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:55.260961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:55.271330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42233c-727f-479c-9895-b336c6384bb9 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:55.319035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ddd5d-2240-4c5b-a6c8-d5bbdcd662d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:55.320803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ddd5d-2240-4c5b-a6c8-d5bbdcd662d5 None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:55.321227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ddd5d-2240-4c5b-a6c8-d5bbdcd662d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:55.321643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ddd5d-2240-4c5b-a6c8-d5bbdcd662d5 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a/users/fcb4950bc93048a891412be785f132b7/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:55.323911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ddd5d-2240-4c5b-a6c8-d5bbdcd662d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d68d17929cf4421f977087a339234c2a, user_id=fcb4950bc93048a891412be785f132b7, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:55.325628 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ddd5d-2240-4c5b-a6c8-d5bbdcd662d5 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:55.344996 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 67/130] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:31:55 2022] PUT /identity/v3/projects/d68d17929cf4421f977087a339234c2a/users/fcb4950bc93048a891412be785f132b7/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:31:55.346125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ddd5d-2240-4c5b-a6c8-d5bbdcd662d5 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:55.389914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824fefe-856a-4259-a5c5-235bb90f3f86 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:55.469531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a339fb7-cd6b-4970-b422-f1503c7f1c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:55.470536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a339fb7-cd6b-4970-b422-f1503c7f1c34 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:55.470965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a339fb7-cd6b-4970-b422-f1503c7f1c34 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:55.471292 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a339fb7-cd6b-4970-b422-f1503c7f1c34 None admin] PATH_INFO: `/v3/users/62833c64bdba4dcfb589eda55e21231f` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:55.498283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ddd5d-2240-4c5b-a6c8-d5bbdcd662d5 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:55.506346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a339fb7-cd6b-4970-b422-f1503c7f1c34 None admin] RBAC: Authorizing `identity:get_user(user_id=62833c64bdba4dcfb589eda55e21231f)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:55.508408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a339fb7-cd6b-4970-b422-f1503c7f1c34 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:55.510376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 64/131] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:55 2022] GET /identity/v3/users/62833c64bdba4dcfb589eda55e21231f => generated 289 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:55.515841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a339fb7-cd6b-4970-b422-f1503c7f1c34 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:55.547357 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887cda2-3389-4e5f-bf1a-e9a8fda47d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:55.549697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887cda2-3389-4e5f-bf1a-e9a8fda47d57 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:55.550475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887cda2-3389-4e5f-bf1a-e9a8fda47d57 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:55.552331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887cda2-3389-4e5f-bf1a-e9a8fda47d57 None admin] PATH_INFO: `/v3/users/9dcd945ca11c4739972e841baad81073` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:55.554137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887cda2-3389-4e5f-bf1a-e9a8fda47d57 None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd945ca11c4739972e841baad81073)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:55.560264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887cda2-3389-4e5f-bf1a-e9a8fda47d57 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:55.562291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 68/132] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:55 2022] GET /identity/v3/users/9dcd945ca11c4739972e841baad81073 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:55.568169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887cda2-3389-4e5f-bf1a-e9a8fda47d57 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:55.604450 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:55.618367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:55.760314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a339fb7-cd6b-4970-b422-f1503c7f1c34 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:55.767721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887cda2-3389-4e5f-bf1a-e9a8fda47d57 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:55.808044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81ee2-ac3f-4890-87c5-e359453c9989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:55.808749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81ee2-ac3f-4890-87c5-e359453c9989 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:55.809039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81ee2-ac3f-4890-87c5-e359453c9989 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:55.809319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81ee2-ac3f-4890-87c5-e359453c9989 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:55.813343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2e47c-7729-487d-8ac7-d0654f223204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:55.814209 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2e47c-7729-487d-8ac7-d0654f223204 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:55.814493 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2e47c-7729-487d-8ac7-d0654f223204 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:55.814772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2e47c-7729-487d-8ac7-d0654f223204 None admin] PATH_INFO: `/v3/projects/5aaef6077b164f09882021c210f75662` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:55.816816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2e47c-7729-487d-8ac7-d0654f223204 None admin] RBAC: Authorizing `identity:get_project(project_id=5aaef6077b164f09882021c210f75662)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:55.818161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2e47c-7729-487d-8ac7-d0654f223204 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:55.819932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 69/133] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:31:55 2022] GET /identity/v3/projects/5aaef6077b164f09882021c210f75662 => generated 312 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:55.833862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2e47c-7729-487d-8ac7-d0654f223204 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:55.936989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81ee2-ac3f-4890-87c5-e359453c9989 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:55.946748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81ee2-ac3f-4890-87c5-e359453c9989 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:55.967563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:55.994616 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2e47c-7729-487d-8ac7-d0654f223204 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:56.011001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:56.036396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377cc25-86ea-4326-8fec-ab9e8e1e90a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:56.037410 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377cc25-86ea-4326-8fec-ab9e8e1e90a6 None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:56.037693 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377cc25-86ea-4326-8fec-ab9e8e1e90a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:56.038032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377cc25-86ea-4326-8fec-ab9e8e1e90a6 None admin] PATH_INFO: `/v3/projects/5aaef6077b164f09882021c210f75662/users/9dcd945ca11c4739972e841baad81073/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:56.040925 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377cc25-86ea-4326-8fec-ab9e8e1e90a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5aaef6077b164f09882021c210f75662, user_id=9dcd945ca11c4739972e841baad81073, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:56.042392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377cc25-86ea-4326-8fec-ab9e8e1e90a6 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:56.071599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 70/134] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:31:55 2022] PUT /identity/v3/projects/5aaef6077b164f09882021c210f75662/users/9dcd945ca11c4739972e841baad81073/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:31:56.091876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 65/135] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:31:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 08:31:56.098003 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377cc25-86ea-4326-8fec-ab9e8e1e90a6 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:56.247458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377cc25-86ea-4326-8fec-ab9e8e1e90a6 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:56.259148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82211d2e-34e7-4cfd-83e6-eaa1e058a418 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:56.259444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82211d2e-34e7-4cfd-83e6-eaa1e058a418 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:56.259727 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82211d2e-34e7-4cfd-83e6-eaa1e058a418 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:56.265701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 66/136] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:56.282451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40930c-a199-4a40-8ab5-0d39f0422456 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:56.282799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40930c-a199-4a40-8ab5-0d39f0422456 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:56.283037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40930c-a199-4a40-8ab5-0d39f0422456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:56.294653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f15190-e765-4c9e-9c7c-2bc37b71b23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:56.295825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f15190-e765-4c9e-9c7c-2bc37b71b23d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:56.296317 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f15190-e765-4c9e-9c7c-2bc37b71b23d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:56.296846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f15190-e765-4c9e-9c7c-2bc37b71b23d None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:56.298324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f15190-e765-4c9e-9c7c-2bc37b71b23d None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:56.300011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f15190-e765-4c9e-9c7c-2bc37b71b23d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:56.302458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 71/137] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:31:56 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:56.308638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f15190-e765-4c9e-9c7c-2bc37b71b23d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:56.333714 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4d40930c-a199-4a40-8ab5-0d39f0422456 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:56.334976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4d40930c-a199-4a40-8ab5-0d39f0422456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:56.336946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:56.452518 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 67/138] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:31:56.481519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40930c-a199-4a40-8ab5-0d39f0422456 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:56.500685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f15190-e765-4c9e-9c7c-2bc37b71b23d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:56.578993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd390267-9059-4526-8288-6c7bfd627479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:56.579850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd390267-9059-4526-8288-6c7bfd627479 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:56.580148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd390267-9059-4526-8288-6c7bfd627479 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:56.580437 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd390267-9059-4526-8288-6c7bfd627479 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:56.581750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd390267-9059-4526-8288-6c7bfd627479 None admin] RBAC: Adding query filter params (role.id=fdcf2d1a66ab4deba04f767592b8c5df, scope.project.id=d68d17929cf4421f977087a339234c2a, user.id=62833c64bdba4dcfb589eda55e21231f) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:56.582076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd390267-9059-4526-8288-6c7bfd627479 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:56.583687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd390267-9059-4526-8288-6c7bfd627479 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:56.597674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 72/139] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:31:56 2022] GET /identity/v3/role_assignments?user.id=62833c64bdba4dcfb589eda55e21231f&scope.project.id=d68d17929cf4421f977087a339234c2a&role.id=fdcf2d1a66ab4deba04f767592b8c5df => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:31:56.599599 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:56.669162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40930c-a199-4a40-8ab5-0d39f0422456 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:56.714423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33034992-4df8-4031-a731-001211318c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:56.715482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33034992-4df8-4031-a731-001211318c5f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:56.715908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33034992-4df8-4031-a731-001211318c5f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:56.716327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33034992-4df8-4031-a731-001211318c5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:56.737828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33034992-4df8-4031-a731-001211318c5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-33034992-4df8-4031-a731-001211318c5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:56.746444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:56.747443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:56.748180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:56.748180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:56.748180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:56.748180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:56.748180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:56.748180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:56.748180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:56.748180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:56.748180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:56.748180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:56.748180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:31:56.748180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:31:56.750185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 68/140] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:31:56.756472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd390267-9059-4526-8288-6c7bfd627479 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:56.863102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:56.892346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd390267-9059-4526-8288-6c7bfd627479 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:56.932424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714cdd2-23ba-479b-bf30-6b853d40391e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:56.933175 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714cdd2-23ba-479b-bf30-6b853d40391e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:56.933466 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714cdd2-23ba-479b-bf30-6b853d40391e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:56.933806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714cdd2-23ba-479b-bf30-6b853d40391e None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:56.934576 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714cdd2-23ba-479b-bf30-6b853d40391e None admin] RBAC: Adding query filter params (name=member) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:56.934888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714cdd2-23ba-479b-bf30-6b853d40391e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:56.936281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714cdd2-23ba-479b-bf30-6b853d40391e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:56.949321 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 73/141] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:31:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:56.954984 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33034992-4df8-4031-a731-001211318c5f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:57.085219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33034992-4df8-4031-a731-001211318c5f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:57.119578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:57.127666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cac533-524e-400f-8f95-c40e070ed7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:57.128555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cac533-524e-400f-8f95-c40e070ed7ce None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:57.128863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cac533-524e-400f-8f95-c40e070ed7ce None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:57.129140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cac533-524e-400f-8f95-c40e070ed7ce None admin] PATH_INFO: `/v3/users/9dcd945ca11c4739972e841baad81073` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:57.130363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cac533-524e-400f-8f95-c40e070ed7ce None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd945ca11c4739972e841baad81073)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:57.131936 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cac533-524e-400f-8f95-c40e070ed7ce None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:57.138548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 69/142] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:56 2022] GET /identity/v3/users/9dcd945ca11c4739972e841baad81073 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:57.143922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714cdd2-23ba-479b-bf30-6b853d40391e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:57.275429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714cdd2-23ba-479b-bf30-6b853d40391e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:57.325810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ba66c-3182-49f6-bd09-2ab56b497546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:57.326829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ba66c-3182-49f6-bd09-2ab56b497546 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:57.327375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ba66c-3182-49f6-bd09-2ab56b497546 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:57.327855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ba66c-3182-49f6-bd09-2ab56b497546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:57.337315 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:57.346919 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fae46-3550-4995-9887-0ceaa8bc7d6e None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:57.347421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fae46-3550-4995-9887-0ceaa8bc7d6e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:57.347918 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fae46-3550-4995-9887-0ceaa8bc7d6e None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:57.349592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 70/143] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:31:57.355011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1368a0d-40cc-4df4-9da8-dea2ee22a500 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:57.355542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1368a0d-40cc-4df4-9da8-dea2ee22a500 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:57.355926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1368a0d-40cc-4df4-9da8-dea2ee22a500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:57.389288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-e1368a0d-40cc-4df4-9da8-dea2ee22a500 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:57.390665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e1368a0d-40cc-4df4-9da8-dea2ee22a500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:57.412901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ba66c-3182-49f6-bd09-2ab56b497546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:57.414732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ba66c-3182-49f6-bd09-2ab56b497546 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:57.535363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 71/144] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:57.574554 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1368a0d-40cc-4df4-9da8-dea2ee22a500 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:57.623371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 74/145] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:31:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:31:57.629007 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ba66c-3182-49f6-bd09-2ab56b497546 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:57.737560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:57.743771 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1368a0d-40cc-4df4-9da8-dea2ee22a500 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:57.772130 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:57.783451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ce53-f3ed-4d11-9f86-46a2e1d2d704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:57.784562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ce53-f3ed-4d11-9f86-46a2e1d2d704 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:57.785102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ce53-f3ed-4d11-9f86-46a2e1d2d704 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:57.785685 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ce53-f3ed-4d11-9f86-46a2e1d2d704 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:57.787551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ce53-f3ed-4d11-9f86-46a2e1d2d704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:57.789955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ce53-f3ed-4d11-9f86-46a2e1d2d704 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:57.791867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 72/146] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:31:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:31:57.797412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ce53-f3ed-4d11-9f86-46a2e1d2d704 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:57.917509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ba66c-3182-49f6-bd09-2ab56b497546 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:57.983642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16201357-1f14-433b-b586-f090f4422600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:57.984517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16201357-1f14-433b-b586-f090f4422600 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:57.984825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16201357-1f14-433b-b586-f090f4422600 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:57.985098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16201357-1f14-433b-b586-f090f4422600 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:57.986992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16201357-1f14-433b-b586-f090f4422600 None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:57.988505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16201357-1f14-433b-b586-f090f4422600 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:57.990199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 75/147] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:31:57 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:31:57.993120 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ce53-f3ed-4d11-9f86-46a2e1d2d704 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:57.996097 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16201357-1f14-433b-b586-f090f4422600 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:58.050583 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5e2a9-ea2e-4502-a793-496f397f4996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:58.051358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5e2a9-ea2e-4502-a793-496f397f4996 None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:58.051645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5e2a9-ea2e-4502-a793-496f397f4996 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:58.051923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5e2a9-ea2e-4502-a793-496f397f4996 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:58.052670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5e2a9-ea2e-4502-a793-496f397f4996 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:58.054312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5e2a9-ea2e-4502-a793-496f397f4996 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:58.086691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 73/148] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:31:57 2022] POST /identity/v3/users => generated 283 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:31:58.087864 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77bdcc-d5a1-47fa-9799-bccee2568f87 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:58.088197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77bdcc-d5a1-47fa-9799-bccee2568f87 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:58.088516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77bdcc-d5a1-47fa-9799-bccee2568f87 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:58.090426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 74/149] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:58.097074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0be290-fecb-4589-951b-b25d079ee573 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:58.097396 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0be290-fecb-4589-951b-b25d079ee573 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:58.097642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0be290-fecb-4589-951b-b25d079ee573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:58.120943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-0c0be290-fecb-4589-951b-b25d079ee573 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:58.123320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0c0be290-fecb-4589-951b-b25d079ee573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:58.166911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16201357-1f14-433b-b586-f090f4422600 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:58.179875 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:58.185175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:58.212859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 75/150] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:58.233027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0be290-fecb-4589-951b-b25d079ee573 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:58.245380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05255e6-963d-4841-9273-41dddc35099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:58.246555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05255e6-963d-4841-9273-41dddc35099b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:58.247091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05255e6-963d-4841-9273-41dddc35099b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:58.247604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05255e6-963d-4841-9273-41dddc35099b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:58.248807 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05255e6-963d-4841-9273-41dddc35099b None admin] RBAC: Adding query filter params (role.id=82ee29c384454b9aa4db085c9d2e21e8, scope.project.id=d68d17929cf4421f977087a339234c2a, user.id=9dcd945ca11c4739972e841baad81073) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:58.249476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05255e6-963d-4841-9273-41dddc35099b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:58.251314 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05255e6-963d-4841-9273-41dddc35099b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:58.263417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 76/151] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:31:57 2022] GET /identity/v3/role_assignments?user.id=9dcd945ca11c4739972e841baad81073&scope.project.id=d68d17929cf4421f977087a339234c2a&role.id=82ee29c384454b9aa4db085c9d2e21e8 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:58.371991 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0be290-fecb-4589-951b-b25d079ee573 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:58.417329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e5a06-973e-4dca-8821-d94eb8fea176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:58.418171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e5a06-973e-4dca-8821-d94eb8fea176 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:58.418461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e5a06-973e-4dca-8821-d94eb8fea176 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:58.418743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e5a06-973e-4dca-8821-d94eb8fea176 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:58.434615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e5a06-973e-4dca-8821-d94eb8fea176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-0c0e5a06-973e-4dca-8821-d94eb8fea176 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:58.443174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:58.444063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:31:58.445439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 76/152] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:31:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:58.451188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05255e6-963d-4841-9273-41dddc35099b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:58.562511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05255e6-963d-4841-9273-41dddc35099b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:58.601217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d4ceb-c0f0-4da7-ba58-64aada75fabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:58.602014 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d4ceb-c0f0-4da7-ba58-64aada75fabf None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:58.602307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d4ceb-c0f0-4da7-ba58-64aada75fabf None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:58.602590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d4ceb-c0f0-4da7-ba58-64aada75fabf None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:58.603421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d4ceb-c0f0-4da7-ba58-64aada75fabf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:58.603735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d4ceb-c0f0-4da7-ba58-64aada75fabf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:58.605214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d4ceb-c0f0-4da7-ba58-64aada75fabf None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:58.615022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 77/153] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:31:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:58.620835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e5a06-973e-4dca-8821-d94eb8fea176 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:58.668966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:58.742385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e5a06-973e-4dca-8821-d94eb8fea176 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:58.785771 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a819a-d9ec-4c0a-a5e0-99d183252c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:58.786672 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a819a-d9ec-4c0a-a5e0-99d183252c66 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:58.786962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a819a-d9ec-4c0a-a5e0-99d183252c66 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:58.787242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a819a-d9ec-4c0a-a5e0-99d183252c66 None admin] PATH_INFO: `/v3/users/9dcd945ca11c4739972e841baad81073` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:58.788516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a819a-d9ec-4c0a-a5e0-99d183252c66 None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd945ca11c4739972e841baad81073)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:58.791341 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc72683-3aea-4597-b8fd-2c2262d01abf None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:58.791636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc72683-3aea-4597-b8fd-2c2262d01abf None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:58.791920 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc72683-3aea-4597-b8fd-2c2262d01abf None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:58.792431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a819a-d9ec-4c0a-a5e0-99d183252c66 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:58.794172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 77/154] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:58 2022] GET /identity/v3/users/9dcd945ca11c4739972e841baad81073 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:58.796670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 78/155] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:31:58.802082 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cfe3b-c324-4ecd-9fd4-694b5a3fbd34 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:58.802377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cfe3b-c324-4ecd-9fd4-694b5a3fbd34 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:58.802660 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cfe3b-c324-4ecd-9fd4-694b5a3fbd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:58.808034 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc72683-3aea-4597-b8fd-2c2262d01abf None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:58.836549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-964cfe3b-c324-4ecd-9fd4-694b5a3fbd34 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:58.838137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-964cfe3b-c324-4ecd-9fd4-694b5a3fbd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:58.857251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:58.935388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 78/156] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:58.951546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc72683-3aea-4597-b8fd-2c2262d01abf None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:58.964876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cfe3b-c324-4ecd-9fd4-694b5a3fbd34 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:58.999053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd8fda-26d2-4292-b1cf-ef6f49d7ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:58.999946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd8fda-26d2-4292-b1cf-ef6f49d7ba3c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:59.000421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd8fda-26d2-4292-b1cf-ef6f49d7ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:59.000881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd8fda-26d2-4292-b1cf-ef6f49d7ba3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:59.016158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:59.091940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd8fda-26d2-4292-b1cf-ef6f49d7ba3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:59.093585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd8fda-26d2-4292-b1cf-ef6f49d7ba3c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:59.095070 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cfe3b-c324-4ecd-9fd4-694b5a3fbd34 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:59.127305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:59.141438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cce55-1a06-4a27-9808-09903a41645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:59.142257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cce55-1a06-4a27-9808-09903a41645c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:59.142545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cce55-1a06-4a27-9808-09903a41645c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:59.142841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cce55-1a06-4a27-9808-09903a41645c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:59.161439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cce55-1a06-4a27-9808-09903a41645c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-171cce55-1a06-4a27-9808-09903a41645c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:59.173422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:59.174403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:59.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:59.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:59.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:59.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:59.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:59.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:59.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:59.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:59.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:59.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:59.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:31:59.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:31:59.178225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 79/157] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:31:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:59.180590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ca4b3-06bf-49b5-9a2b-903dc810fe7e None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:59.181094 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ca4b3-06bf-49b5-9a2b-903dc810fe7e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:59.181584 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ca4b3-06bf-49b5-9a2b-903dc810fe7e None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:59.184242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 80/158] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:59.190594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5f629-b0fd-4390-8576-4741a9d9dcab None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:59.190756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5f629-b0fd-4390-8576-4741a9d9dcab None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:59.190972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5f629-b0fd-4390-8576-4741a9d9dcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:59.217102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-0ea5f629-b0fd-4390-8576-4741a9d9dcab None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:31:59.219583 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0ea5f629-b0fd-4390-8576-4741a9d9dcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:31:59.235283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 79/159] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:31:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 08:31:59.236227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd8fda-26d2-4292-b1cf-ef6f49d7ba3c None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:59.337429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 81/160] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:31:59.338689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5f629-b0fd-4390-8576-4741a9d9dcab None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:59.377802 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:59.398862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd8fda-26d2-4292-b1cf-ef6f49d7ba3c None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:59.445803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:59.453450 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38bed0-a51f-4779-b886-56bdd824058c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:59.454731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38bed0-a51f-4779-b886-56bdd824058c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:59.455168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38bed0-a51f-4779-b886-56bdd824058c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:59.455574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38bed0-a51f-4779-b886-56bdd824058c None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:59.457006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38bed0-a51f-4779-b886-56bdd824058c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:31:59.457479 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38bed0-a51f-4779-b886-56bdd824058c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:59.459765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38bed0-a51f-4779-b886-56bdd824058c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:59.474023 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 80/161] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:59.474763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38bed0-a51f-4779-b886-56bdd824058c None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:59.490414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5f629-b0fd-4390-8576-4741a9d9dcab None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:59.559309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f69c5-82a3-44e9-a285-aefdd465bcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:59.560203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f69c5-82a3-44e9-a285-aefdd465bcb4 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:59.560488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f69c5-82a3-44e9-a285-aefdd465bcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:59.560793 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f69c5-82a3-44e9-a285-aefdd465bcb4 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:59.565423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f69c5-82a3-44e9-a285-aefdd465bcb4 None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:59.570193 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f69c5-82a3-44e9-a285-aefdd465bcb4 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:59.572207 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 82/162] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:31:59 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:59.572807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f69c5-82a3-44e9-a285-aefdd465bcb4 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:59.654048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38bed0-a51f-4779-b886-56bdd824058c None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:59.705648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88004e0e-29fa-437f-86d3-46f85b007fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:59.706462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88004e0e-29fa-437f-86d3-46f85b007fb0 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:59.706744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88004e0e-29fa-437f-86d3-46f85b007fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:59.707021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88004e0e-29fa-437f-86d3-46f85b007fb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:59.708471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:59.720157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f69c5-82a3-44e9-a285-aefdd465bcb4 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:59.732255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88004e0e-29fa-437f-86d3-46f85b007fb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-88004e0e-29fa-437f-86d3-46f85b007fb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:31:59.741016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:31:59.741954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:31:59.742729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:31:59.742729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:31:59.742729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:31:59.742729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:31:59.742729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:31:59.742729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:31:59.742729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:31:59.742729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:31:59.742729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:31:59.742729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:31:59.742729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:31:59.742729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:31:59.743364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 81/163] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:31:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:31:59.744041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88004e0e-29fa-437f-86d3-46f85b007fb0 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:59.783095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba3812-b638-405a-b43f-65c29333f01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:31:59.784726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba3812-b638-405a-b43f-65c29333f01b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:59.785013 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba3812-b638-405a-b43f-65c29333f01b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:59.785291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba3812-b638-405a-b43f-65c29333f01b None admin] PATH_INFO: `/v3/users/fcb4950bc93048a891412be785f132b7` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:59.790164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba3812-b638-405a-b43f-65c29333f01b None admin] RBAC: Authorizing `identity:get_user(user_id=fcb4950bc93048a891412be785f132b7)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:31:59.791693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba3812-b638-405a-b43f-65c29333f01b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:31:59.797854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 83/164] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:31:59 2022] GET /identity/v3/users/fcb4950bc93048a891412be785f132b7 => generated 247 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:31:59.799133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899656fc-313c-4360-8a41-695ab76634f8 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:31:59.799428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899656fc-313c-4360-8a41-695ab76634f8 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:31:59.799675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899656fc-313c-4360-8a41-695ab76634f8 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:31:59.801794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 84/165] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:31:59.802512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899656fc-313c-4360-8a41-695ab76634f8 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:31:59.853327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:31:59.960943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88004e0e-29fa-437f-86d3-46f85b007fb0 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:31:59.972488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899656fc-313c-4360-8a41-695ab76634f8 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:00.012102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee143a-3d0a-4fe8-9130-daf6e1c7f876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:00.013954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee143a-3d0a-4fe8-9130-daf6e1c7f876 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:00.014239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee143a-3d0a-4fe8-9130-daf6e1c7f876 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:00.014517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee143a-3d0a-4fe8-9130-daf6e1c7f876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:00.016916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee143a-3d0a-4fe8-9130-daf6e1c7f876 None admin] RBAC: Adding query filter params (role.id=3ce6c4ec0ed142828cb20882e342328e, scope.project.id=d68d17929cf4421f977087a339234c2a, user.id=9dcd945ca11c4739972e841baad81073) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:00.017242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee143a-3d0a-4fe8-9130-daf6e1c7f876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:00.020622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee143a-3d0a-4fe8-9130-daf6e1c7f876 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:00.034291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 82/166] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:31:59 2022] GET /identity/v3/role_assignments?user.id=9dcd945ca11c4739972e841baad81073&scope.project.id=d68d17929cf4421f977087a339234c2a&role.id=3ce6c4ec0ed142828cb20882e342328e => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:00.034967 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee143a-3d0a-4fe8-9130-daf6e1c7f876 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:00.041489 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf8c7e-f52f-4f9c-ae3a-04bfd21a85e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:00.042325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf8c7e-f52f-4f9c-ae3a-04bfd21a85e2 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:00.042612 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf8c7e-f52f-4f9c-ae3a-04bfd21a85e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:00.042890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf8c7e-f52f-4f9c-ae3a-04bfd21a85e2 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:00.044953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf8c7e-f52f-4f9c-ae3a-04bfd21a85e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:00.045268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf8c7e-f52f-4f9c-ae3a-04bfd21a85e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:00.046706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf8c7e-f52f-4f9c-ae3a-04bfd21a85e2 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:00.065981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 85/167] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:31:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:00.067584 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d8c5c-2d95-46c8-ab04-43a8e6f13a41 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:00.067879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d8c5c-2d95-46c8-ab04-43a8e6f13a41 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:00.068159 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d8c5c-2d95-46c8-ab04-43a8e6f13a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:00.109141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-e20d8c5c-2d95-46c8-ab04-43a8e6f13a41 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:00.111218 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e20d8c5c-2d95-46c8-ab04-43a8e6f13a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:00.150572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:00.188070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee143a-3d0a-4fe8-9130-daf6e1c7f876 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:00.234729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:00.267288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1bb60-abad-4097-949f-da166db44aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:00.267955 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1bb60-abad-4097-949f-da166db44aa1 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:00.268238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1bb60-abad-4097-949f-da166db44aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:00.268512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1bb60-abad-4097-949f-da166db44aa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:00.288008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 86/168] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:00.288894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d8c5c-2d95-46c8-ab04-43a8e6f13a41 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:00.398658 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1bb60-abad-4097-949f-da166db44aa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:00.400140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1bb60-abad-4097-949f-da166db44aa1 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:00.476280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d8c5c-2d95-46c8-ab04-43a8e6f13a41 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:00.546545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86bbb8-77af-4a44-87d5-17cb023c9f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:00.547358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86bbb8-77af-4a44-87d5-17cb023c9f8d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:00.547646 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86bbb8-77af-4a44-87d5-17cb023c9f8d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:00.547926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86bbb8-77af-4a44-87d5-17cb023c9f8d None admin] PATH_INFO: `/v3/users/9dcd945ca11c4739972e841baad81073` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:00.553300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86bbb8-77af-4a44-87d5-17cb023c9f8d None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd945ca11c4739972e841baad81073)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:00.555339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86bbb8-77af-4a44-87d5-17cb023c9f8d None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:00.556401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 83/169] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:00.557144 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 87/170] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:00 2022] GET /identity/v3/users/9dcd945ca11c4739972e841baad81073 => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:00.560725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1bb60-abad-4097-949f-da166db44aa1 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:00.562585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86bbb8-77af-4a44-87d5-17cb023c9f8d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:00.584344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:00.621933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:00.687626 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1bb60-abad-4097-949f-da166db44aa1 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:00.751984 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580b207-3113-44c8-877a-259405f72fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:00.752816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580b207-3113-44c8-877a-259405f72fd9 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:00.753097 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580b207-3113-44c8-877a-259405f72fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:00.753369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580b207-3113-44c8-877a-259405f72fd9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:00.780254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580b207-3113-44c8-877a-259405f72fd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:00.784351 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86bbb8-77af-4a44-87d5-17cb023c9f8d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-5580b207-3113-44c8-877a-259405f72fd9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:00.800182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:00.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:00.801897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 84/171] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:00.802643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580b207-3113-44c8-877a-259405f72fd9 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:00.832491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40bfbd-7fff-4975-b1a1-a8f5470efd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:00.833392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40bfbd-7fff-4975-b1a1-a8f5470efd90 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:00.833675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40bfbd-7fff-4975-b1a1-a8f5470efd90 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:00.833947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40bfbd-7fff-4975-b1a1-a8f5470efd90 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:00.836247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40bfbd-7fff-4975-b1a1-a8f5470efd90 None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:00.837739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40bfbd-7fff-4975-b1a1-a8f5470efd90 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:00.840706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 88/172] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:00 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:00.842327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40bfbd-7fff-4975-b1a1-a8f5470efd90 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:00.964028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:00.974414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40bfbd-7fff-4975-b1a1-a8f5470efd90 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:00.980591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:01.011759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580b207-3113-44c8-877a-259405f72fd9 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:01.016594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5e488-3112-4773-b8f8-e5e2d593d4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:01.016594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5e488-3112-4773-b8f8-e5e2d593d4a8 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:01.016594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5e488-3112-4773-b8f8-e5e2d593d4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:01.016594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5e488-3112-4773-b8f8-e5e2d593d4a8 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:01.016946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5e488-3112-4773-b8f8-e5e2d593d4a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:01.016946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5e488-3112-4773-b8f8-e5e2d593d4a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:01.018327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5e488-3112-4773-b8f8-e5e2d593d4a8 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:01.032975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 89/173] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:01.032975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5e488-3112-4773-b8f8-e5e2d593d4a8 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:01.096161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af34975-160a-40b0-8392-dfd40774dd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:01.096776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af34975-160a-40b0-8392-dfd40774dd1c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:01.097068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af34975-160a-40b0-8392-dfd40774dd1c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:01.097351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af34975-160a-40b0-8392-dfd40774dd1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:01.140301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5e488-3112-4773-b8f8-e5e2d593d4a8 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:01.182536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38274e08-df3b-4700-9ce6-92db37415496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:01.183200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38274e08-df3b-4700-9ce6-92db37415496 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:01.183483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38274e08-df3b-4700-9ce6-92db37415496 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:01.183759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38274e08-df3b-4700-9ce6-92db37415496 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:01.184613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38274e08-df3b-4700-9ce6-92db37415496 None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=d68d17929cf4421f977087a339234c2a, user.id=fcb4950bc93048a891412be785f132b7) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:01.184933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38274e08-df3b-4700-9ce6-92db37415496 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:01.186370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38274e08-df3b-4700-9ce6-92db37415496 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:01.200820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 90/174] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:01 2022] GET /identity/v3/role_assignments?user.id=fcb4950bc93048a891412be785f132b7&scope.project.id=d68d17929cf4421f977087a339234c2a&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:01.201791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38274e08-df3b-4700-9ce6-92db37415496 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:01.223918 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:01.248094 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af34975-160a-40b0-8392-dfd40774dd1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:01.249967 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af34975-160a-40b0-8392-dfd40774dd1c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:01.320886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38274e08-df3b-4700-9ce6-92db37415496 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:01.359880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca259fc-3616-4d8d-b2ec-3da90f83ec52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:01.360715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca259fc-3616-4d8d-b2ec-3da90f83ec52 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:01.361000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca259fc-3616-4d8d-b2ec-3da90f83ec52 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:01.361275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca259fc-3616-4d8d-b2ec-3da90f83ec52 None admin] PATH_INFO: `/v3/users/62833c64bdba4dcfb589eda55e21231f` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:01.362492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca259fc-3616-4d8d-b2ec-3da90f83ec52 None admin] RBAC: Authorizing `identity:get_user(user_id=62833c64bdba4dcfb589eda55e21231f)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:01.363948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca259fc-3616-4d8d-b2ec-3da90f83ec52 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:01.365800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 91/175] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:01 2022] GET /identity/v3/users/62833c64bdba4dcfb589eda55e21231f => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:01.373789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca259fc-3616-4d8d-b2ec-3da90f83ec52 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:01.407009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:01.480810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:01.495548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 85/176] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:01.496824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca259fc-3616-4d8d-b2ec-3da90f83ec52 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:01.496937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7bba4-2e99-41d0-9d88-d080172b2b11 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:01.497210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7bba4-2e99-41d0-9d88-d080172b2b11 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:01.497489 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7bba4-2e99-41d0-9d88-d080172b2b11 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:01.498785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 86/177] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:01.503460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7bba4-2e99-41d0-9d88-d080172b2b11 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:01.547148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763bb0f-f98f-49e7-bf4e-60ed76dee04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:01.548363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763bb0f-f98f-49e7-bf4e-60ed76dee04e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:01.548898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763bb0f-f98f-49e7-bf4e-60ed76dee04e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:01.549569 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763bb0f-f98f-49e7-bf4e-60ed76dee04e None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:01.551862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763bb0f-f98f-49e7-bf4e-60ed76dee04e None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:01.553591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763bb0f-f98f-49e7-bf4e-60ed76dee04e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:01.556120 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 92/178] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:01 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:01.557718 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514d8d3-9d24-4134-9574-eaf4b2d8d8e5 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:01.558729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514d8d3-9d24-4134-9574-eaf4b2d8d8e5 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:01.559204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514d8d3-9d24-4134-9574-eaf4b2d8d8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:01.586728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c514d8d3-9d24-4134-9574-eaf4b2d8d8e5 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:01.587521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c514d8d3-9d24-4134-9574-eaf4b2d8d8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:01.638380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7bba4-2e99-41d0-9d88-d080172b2b11 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:01.672126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 93/179] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:01.676593 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514d8d3-9d24-4134-9574-eaf4b2d8d8e5 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:01.684881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bbdab-db19-440d-bff1-7a6c7b4fd73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:01.686110 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bbdab-db19-440d-bff1-7a6c7b4fd73c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:01.686614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bbdab-db19-440d-bff1-7a6c7b4fd73c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:01.687093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bbdab-db19-440d-bff1-7a6c7b4fd73c None admin] PATH_INFO: `/v3/projects/5aaef6077b164f09882021c210f75662` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:01.688536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bbdab-db19-440d-bff1-7a6c7b4fd73c None admin] RBAC: Authorizing `identity:get_project(project_id=5aaef6077b164f09882021c210f75662)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:01.694420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bbdab-db19-440d-bff1-7a6c7b4fd73c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:01.697085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 87/180] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:01 2022] GET /identity/v3/projects/5aaef6077b164f09882021c210f75662 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:01.710933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bbdab-db19-440d-bff1-7a6c7b4fd73c None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:01.771874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:01.779524 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:01.798716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514d8d3-9d24-4134-9574-eaf4b2d8d8e5 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:01.835371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08aad9-c1ef-424a-9cad-b98ad60e3289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:01.836276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08aad9-c1ef-424a-9cad-b98ad60e3289 None admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:01.836561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08aad9-c1ef-424a-9cad-b98ad60e3289 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:01.836866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08aad9-c1ef-424a-9cad-b98ad60e3289 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a/users/62833c64bdba4dcfb589eda55e21231f/roles/fdcf2d1a66ab4deba04f767592b8c5df` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:01.839085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08aad9-c1ef-424a-9cad-b98ad60e3289 None admin] RBAC: Authorizing `identity:create_grant(project_id=d68d17929cf4421f977087a339234c2a, user_id=62833c64bdba4dcfb589eda55e21231f, role_id=fdcf2d1a66ab4deba04f767592b8c5df)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:01.840539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08aad9-c1ef-424a-9cad-b98ad60e3289 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:01.840841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bbdab-db19-440d-bff1-7a6c7b4fd73c None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:01.864603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 94/181] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:32:01 2022] PUT /identity/v3/projects/d68d17929cf4421f977087a339234c2a/users/62833c64bdba4dcfb589eda55e21231f/roles/fdcf2d1a66ab4deba04f767592b8c5df => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:01.864603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08aad9-c1ef-424a-9cad-b98ad60e3289 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:01.903496 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89983a22-b8e2-4f8f-87f4-d2493f0b4dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:01.904238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89983a22-b8e2-4f8f-87f4-d2493f0b4dea None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:01.904519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89983a22-b8e2-4f8f-87f4-d2493f0b4dea None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:01.904817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89983a22-b8e2-4f8f-87f4-d2493f0b4dea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:01.925303 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89983a22-b8e2-4f8f-87f4-d2493f0b4dea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-89983a22-b8e2-4f8f-87f4-d2493f0b4dea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:01.937374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:01.938326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:01.939103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:01.939103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:01.939103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:01.939103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:01.939103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:01.939103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:01.939103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:01.939103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:01.939103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:01.939103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:01.939103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:01.939103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:01.941778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 88/182] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:32:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:01.947606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231141ea-3750-4d95-b93e-e37637acc21c None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:01.947606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231141ea-3750-4d95-b93e-e37637acc21c None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:01.947741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231141ea-3750-4d95-b93e-e37637acc21c None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:01.949410 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 89/183] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:01.950112 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231141ea-3750-4d95-b93e-e37637acc21c None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:02.002129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08aad9-c1ef-424a-9cad-b98ad60e3289 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:02.053191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ce6c3-b0bd-4556-ab06-a1242c3cb240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:02.054757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ce6c3-b0bd-4556-ab06-a1242c3cb240 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:02.055242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ce6c3-b0bd-4556-ab06-a1242c3cb240 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:02.055768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ce6c3-b0bd-4556-ab06-a1242c3cb240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:02.056991 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ce6c3-b0bd-4556-ab06-a1242c3cb240 None admin] RBAC: Adding query filter params (role.id=82ee29c384454b9aa4db085c9d2e21e8, scope.project.id=5aaef6077b164f09882021c210f75662, user.id=9dcd945ca11c4739972e841baad81073) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:02.057520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ce6c3-b0bd-4556-ab06-a1242c3cb240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:02.065092 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ce6c3-b0bd-4556-ab06-a1242c3cb240 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:02.072137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231141ea-3750-4d95-b93e-e37637acc21c None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:02.072661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 95/184] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:01 2022] GET /identity/v3/role_assignments?user.id=9dcd945ca11c4739972e841baad81073&scope.project.id=5aaef6077b164f09882021c210f75662&role.id=82ee29c384454b9aa4db085c9d2e21e8 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:02.074078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d99231-cb81-49c5-ba79-475859059eb9 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:02.074367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d99231-cb81-49c5-ba79-475859059eb9 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:02.074650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d99231-cb81-49c5-ba79-475859059eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:02.089070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:02.095249 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-20d99231-cb81-49c5-ba79-475859059eb9 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:02.096755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-20d99231-cb81-49c5-ba79-475859059eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:02.101846 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:02.118723 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87a4d6-8253-466c-9264-ac1946bc6dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:02.119507 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87a4d6-8253-466c-9264-ac1946bc6dd2 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:02.119865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87a4d6-8253-466c-9264-ac1946bc6dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:02.120210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87a4d6-8253-466c-9264-ac1946bc6dd2 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:02.121096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87a4d6-8253-466c-9264-ac1946bc6dd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:02.121476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87a4d6-8253-466c-9264-ac1946bc6dd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:02.122985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87a4d6-8253-466c-9264-ac1946bc6dd2 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:02.132277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 90/185] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:02.137142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be06b97-1d6c-4fa9-b974-b15700e89f89 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:02.137142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be06b97-1d6c-4fa9-b974-b15700e89f89 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:02.137142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be06b97-1d6c-4fa9-b974-b15700e89f89 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:02.137142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 91/186] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:02.140752 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be06b97-1d6c-4fa9-b974-b15700e89f89 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:02.189992 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 96/187] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:02.191758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae72f41-3b0b-49a2-8931-b02c321f0d58 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:02.192225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae72f41-3b0b-49a2-8931-b02c321f0d58 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:02.192701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae72f41-3b0b-49a2-8931-b02c321f0d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:02.212697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-9ae72f41-3b0b-49a2-8931-b02c321f0d58 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:02.214309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9ae72f41-3b0b-49a2-8931-b02c321f0d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:02.253763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be06b97-1d6c-4fa9-b974-b15700e89f89 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:02.296318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01e22d-fc84-4b4a-a9e1-172b85132035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:02.297576 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01e22d-fc84-4b4a-a9e1-172b85132035 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:02.298020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01e22d-fc84-4b4a-a9e1-172b85132035 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:02.298455 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01e22d-fc84-4b4a-a9e1-172b85132035 None admin] PATH_INFO: `/v3/users/64b59173e29c415d80ea6813088f3f22` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:02.300126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 97/188] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:02.303303 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae72f41-3b0b-49a2-8931-b02c321f0d58 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:02.321413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01e22d-fc84-4b4a-a9e1-172b85132035 None admin] RBAC: Authorizing `identity:get_user(user_id=64b59173e29c415d80ea6813088f3f22)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:02.323164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01e22d-fc84-4b4a-a9e1-172b85132035 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:02.325814 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 92/189] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:02 2022] GET /identity/v3/users/64b59173e29c415d80ea6813088f3f22 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:02.331472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01e22d-fc84-4b4a-a9e1-172b85132035 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:02.367302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:02.384617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:02.428703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae72f41-3b0b-49a2-8931-b02c321f0d58 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:02.458387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01e22d-fc84-4b4a-a9e1-172b85132035 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:02.473080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1797278-6e42-40bb-af07-0c348768c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:02.473909 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1797278-6e42-40bb-af07-0c348768c79c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:02.474183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1797278-6e42-40bb-af07-0c348768c79c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:02.474456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1797278-6e42-40bb-af07-0c348768c79c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:02.486301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1797278-6e42-40bb-af07-0c348768c79c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-c1797278-6e42-40bb-af07-0c348768c79c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:02.494639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:02.495746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:02.496753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:02.496753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:02.496753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:02.496753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:02.496753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:02.496753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:02.496753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:02.496753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:02.496753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:02.496753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:02.496753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:02.496753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:02.498668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95e765-63ca-4b31-9839-10585be69efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:02.498795 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 98/190] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:32:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:02.499222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95e765-63ca-4b31-9839-10585be69efa None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:02.499573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95e765-63ca-4b31-9839-10585be69efa None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:02.499669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1797278-6e42-40bb-af07-0c348768c79c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:02.499851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95e765-63ca-4b31-9839-10585be69efa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:02.577528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95e765-63ca-4b31-9839-10585be69efa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:02.579051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95e765-63ca-4b31-9839-10585be69efa None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:02.606967 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:02.614371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1797278-6e42-40bb-af07-0c348768c79c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:02.642245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:02.651650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745eccb-8236-4271-9300-0a6b6a708162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:02.652440 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745eccb-8236-4271-9300-0a6b6a708162 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:02.652736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745eccb-8236-4271-9300-0a6b6a708162 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:02.653021 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745eccb-8236-4271-9300-0a6b6a708162 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:02.654213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745eccb-8236-4271-9300-0a6b6a708162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:02.656023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745eccb-8236-4271-9300-0a6b6a708162 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:02.657701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 99/191] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:02.658475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745eccb-8236-4271-9300-0a6b6a708162 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:02.686468 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 93/192] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 08:32:02.687536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95e765-63ca-4b31-9839-10585be69efa None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:02.772260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745eccb-8236-4271-9300-0a6b6a708162 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:02.800358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95e765-63ca-4b31-9839-10585be69efa None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:02.808466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c6c5c-fdf4-427e-a31c-581fc3288240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:02.809192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c6c5c-fdf4-427e-a31c-581fc3288240 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:02.809483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c6c5c-fdf4-427e-a31c-581fc3288240 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:02.809766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c6c5c-fdf4-427e-a31c-581fc3288240 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:02.810525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c6c5c-fdf4-427e-a31c-581fc3288240 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:02.810837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c6c5c-fdf4-427e-a31c-581fc3288240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:02.812217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c6c5c-fdf4-427e-a31c-581fc3288240 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:02.825469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 100/193] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:02.825469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c6c5c-fdf4-427e-a31c-581fc3288240 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:02.832344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:02.844011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ae221-83b9-41a7-b619-ba7fa479cb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:02.845184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ae221-83b9-41a7-b619-ba7fa479cb2f None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:02.845480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ae221-83b9-41a7-b619-ba7fa479cb2f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:02.845758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ae221-83b9-41a7-b619-ba7fa479cb2f None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:02.846645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ae221-83b9-41a7-b619-ba7fa479cb2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:02.848113 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ae221-83b9-41a7-b619-ba7fa479cb2f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:02.870409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 94/194] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:32:02 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:32:02.871547 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ae221-83b9-41a7-b619-ba7fa479cb2f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:02.896970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:02.963749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c6c5c-fdf4-427e-a31c-581fc3288240 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:02.994164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ae221-83b9-41a7-b619-ba7fa479cb2f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:03.006254 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0e765-b838-4d77-96e7-81224cbaaa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:03.008106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0e765-b838-4d77-96e7-81224cbaaa1f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:03.008628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0e765-b838-4d77-96e7-81224cbaaa1f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:03.009113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0e765-b838-4d77-96e7-81224cbaaa1f None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:03.010770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0e765-b838-4d77-96e7-81224cbaaa1f None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:03.012551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0e765-b838-4d77-96e7-81224cbaaa1f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:03.014751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 101/195] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:02 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:03.021480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0e765-b838-4d77-96e7-81224cbaaa1f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:03.038926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd22225-c9a4-48a3-b42c-23d59344a25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:03.039733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd22225-c9a4-48a3-b42c-23d59344a25c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:03.040020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd22225-c9a4-48a3-b42c-23d59344a25c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:03.040298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd22225-c9a4-48a3-b42c-23d59344a25c None admin] PATH_INFO: `/v3/users/64b59173e29c415d80ea6813088f3f22` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:03.041569 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd22225-c9a4-48a3-b42c-23d59344a25c None admin] RBAC: Authorizing `identity:get_user(user_id=64b59173e29c415d80ea6813088f3f22)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:03.043027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd22225-c9a4-48a3-b42c-23d59344a25c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:03.045347 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 95/196] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:02 2022] GET /identity/v3/users/64b59173e29c415d80ea6813088f3f22 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:03.052483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd22225-c9a4-48a3-b42c-23d59344a25c None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:03.119591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:03.157608 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0e765-b838-4d77-96e7-81224cbaaa1f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:03.191840 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:03.195610 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd22225-c9a4-48a3-b42c-23d59344a25c None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:03.202292 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26293f-6d9e-4265-ac4c-dcb8b15d0adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:03.202972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26293f-6d9e-4265-ac4c-dcb8b15d0adb None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:03.203256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26293f-6d9e-4265-ac4c-dcb8b15d0adb None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:03.203538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26293f-6d9e-4265-ac4c-dcb8b15d0adb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:03.205789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26293f-6d9e-4265-ac4c-dcb8b15d0adb None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb, user.id=64b59173e29c415d80ea6813088f3f22) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:03.205789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26293f-6d9e-4265-ac4c-dcb8b15d0adb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:03.206866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26293f-6d9e-4265-ac4c-dcb8b15d0adb None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:03.215291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 102/197] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:03 2022] GET /identity/v3/role_assignments?user.id=64b59173e29c415d80ea6813088f3f22&scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:03.239171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920481b-dbac-4af5-8828-89566a715f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:03.240298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920481b-dbac-4af5-8828-89566a715f21 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:03.240842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920481b-dbac-4af5-8828-89566a715f21 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:03.241867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920481b-dbac-4af5-8828-89566a715f21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:03.321302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920481b-dbac-4af5-8828-89566a715f21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:03.322874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920481b-dbac-4af5-8828-89566a715f21 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:03.377911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:03.434374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 96/198] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 08:32:03.439989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26293f-6d9e-4265-ac4c-dcb8b15d0adb None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:03.588067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26293f-6d9e-4265-ac4c-dcb8b15d0adb None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:03.624044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a27964-ede1-41cc-a868-832541b0db66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:03.625396 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a27964-ede1-41cc-a868-832541b0db66 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:03.625879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a27964-ede1-41cc-a868-832541b0db66 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:03.626349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a27964-ede1-41cc-a868-832541b0db66 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:03.627727 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a27964-ede1-41cc-a868-832541b0db66 None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:03.629523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a27964-ede1-41cc-a868-832541b0db66 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:03.631448 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 103/199] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:03 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:03.639493 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920481b-dbac-4af5-8828-89566a715f21 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:03.796788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920481b-dbac-4af5-8828-89566a715f21 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:03.832280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a1870-b983-4f4b-899b-0b5f50a15281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:03.832987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a1870-b983-4f4b-899b-0b5f50a15281 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:03.833275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a1870-b983-4f4b-899b-0b5f50a15281 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:03.833555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a1870-b983-4f4b-899b-0b5f50a15281 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:03.834405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a1870-b983-4f4b-899b-0b5f50a15281 None admin] RBAC: Adding query filter params (role.id=3ce6c4ec0ed142828cb20882e342328e, scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb, user.id=64b59173e29c415d80ea6813088f3f22) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:03.834723 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a1870-b983-4f4b-899b-0b5f50a15281 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:03.836266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a1870-b983-4f4b-899b-0b5f50a15281 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:03.842379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 97/200] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:03 2022] GET /identity/v3/role_assignments?user.id=64b59173e29c415d80ea6813088f3f22&scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb&role.id=3ce6c4ec0ed142828cb20882e342328e => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:04.035895 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7eb406-aa87-431f-9811-e5d68923c673 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:04.036783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7eb406-aa87-431f-9811-e5d68923c673 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:04.037211 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7eb406-aa87-431f-9811-e5d68923c673 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:04.039361 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 104/201] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:04.044913 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dada901-3da6-4074-8ed7-966cabe96291 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:04.045432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dada901-3da6-4074-8ed7-966cabe96291 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:04.045906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dada901-3da6-4074-8ed7-966cabe96291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:04.065109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-1dada901-3da6-4074-8ed7-966cabe96291 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:04.066595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1dada901-3da6-4074-8ed7-966cabe96291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:04.077100 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:04.138200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 98/202] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:04.161596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7eb406-aa87-431f-9811-e5d68923c673 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:04.223696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:04.268007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7eb406-aa87-431f-9811-e5d68923c673 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:04.316231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bac24-5421-4953-8329-b0c874b80192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:04.317068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bac24-5421-4953-8329-b0c874b80192 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:04.317356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bac24-5421-4953-8329-b0c874b80192 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:04.317629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bac24-5421-4953-8329-b0c874b80192 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:04.328675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bac24-5421-4953-8329-b0c874b80192 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-ce2bac24-5421-4953-8329-b0c874b80192 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:04.336960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:04.337787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:04.339001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 105/203] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:04.343924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dada901-3da6-4074-8ed7-966cabe96291 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:04.444210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dada901-3da6-4074-8ed7-966cabe96291 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:04.479305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2f951-9d6d-4459-a4e4-4bea0ffb59be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:04.479987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2f951-9d6d-4459-a4e4-4bea0ffb59be None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:04.480266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2f951-9d6d-4459-a4e4-4bea0ffb59be None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:04.480537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2f951-9d6d-4459-a4e4-4bea0ffb59be None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:04.481317 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2f951-9d6d-4459-a4e4-4bea0ffb59be None admin] RBAC: Adding query filter params (name=reader) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:04.481621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2f951-9d6d-4459-a4e4-4bea0ffb59be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:04.482999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2f951-9d6d-4459-a4e4-4bea0ffb59be None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:04.491748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 99/204] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:04.497282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bac24-5421-4953-8329-b0c874b80192 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:04.601875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bac24-5421-4953-8329-b0c874b80192 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:04.618861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:04.636043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f2328-f1c3-49c6-ba93-dd1f5d73b9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:04.637019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f2328-f1c3-49c6-ba93-dd1f5d73b9a0 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:04.637311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f2328-f1c3-49c6-ba93-dd1f5d73b9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:04.637594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f2328-f1c3-49c6-ba93-dd1f5d73b9a0 None admin] PATH_INFO: `/v3/users/62833c64bdba4dcfb589eda55e21231f` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:04.638798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f2328-f1c3-49c6-ba93-dd1f5d73b9a0 None admin] RBAC: Authorizing `identity:get_user(user_id=62833c64bdba4dcfb589eda55e21231f)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:04.640439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f2328-f1c3-49c6-ba93-dd1f5d73b9a0 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:04.648161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2f951-9d6d-4459-a4e4-4bea0ffb59be None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:04.660272 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 106/205] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:04 2022] GET /identity/v3/users/62833c64bdba4dcfb589eda55e21231f => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:04.664887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:04.749189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2f951-9d6d-4459-a4e4-4bea0ffb59be None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:04.783369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdb53b-c3c1-4f25-87ca-dddf48a75cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:04.784031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdb53b-c3c1-4f25-87ca-dddf48a75cbc None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:04.784315 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdb53b-c3c1-4f25-87ca-dddf48a75cbc None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:04.784606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdb53b-c3c1-4f25-87ca-dddf48a75cbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:04.853397 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efdb53b-c3c1-4f25-87ca-dddf48a75cbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:04.854868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efdb53b-c3c1-4f25-87ca-dddf48a75cbc None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:04.862794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:04.955165 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 100/206] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:04.960819 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f2328-f1c3-49c6-ba93-dd1f5d73b9a0 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:05.065213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f2328-f1c3-49c6-ba93-dd1f5d73b9a0 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:05.098798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4d118-7c21-4c37-b4af-e16107d8548b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:05.099677 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4d118-7c21-4c37-b4af-e16107d8548b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:05.099964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4d118-7c21-4c37-b4af-e16107d8548b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:05.100242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4d118-7c21-4c37-b4af-e16107d8548b None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:05.101980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4d118-7c21-4c37-b4af-e16107d8548b None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:05.103434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4d118-7c21-4c37-b4af-e16107d8548b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:05.105293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 107/207] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:04 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:05.111100 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdb53b-c3c1-4f25-87ca-dddf48a75cbc None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:05.211293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdb53b-c3c1-4f25-87ca-dddf48a75cbc None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:05.219257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:05.246943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74ae60-f286-488f-a42c-1cb12e846eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:05.247619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74ae60-f286-488f-a42c-1cb12e846eb3 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:05.247900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74ae60-f286-488f-a42c-1cb12e846eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:05.248173 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74ae60-f286-488f-a42c-1cb12e846eb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:05.249032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74ae60-f286-488f-a42c-1cb12e846eb3 None admin] RBAC: Adding query filter params (role.id=fdcf2d1a66ab4deba04f767592b8c5df, scope.project.id=d68d17929cf4421f977087a339234c2a, user.id=62833c64bdba4dcfb589eda55e21231f) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:05.249342 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74ae60-f286-488f-a42c-1cb12e846eb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:05.250788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74ae60-f286-488f-a42c-1cb12e846eb3 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:05.256848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 101/208] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:05 2022] GET /identity/v3/role_assignments?user.id=62833c64bdba4dcfb589eda55e21231f&scope.project.id=d68d17929cf4421f977087a339234c2a&role.id=fdcf2d1a66ab4deba04f767592b8c5df => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:05.596424 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6c2c2-bcfb-4449-bbd0-303880fa57fc None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:05.596767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6c2c2-bcfb-4449-bbd0-303880fa57fc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:05.597131 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6c2c2-bcfb-4449-bbd0-303880fa57fc None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:05.598792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 108/209] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:05.604322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411de996-a020-41fb-8afa-b3b922c7812e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:05.604640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411de996-a020-41fb-8afa-b3b922c7812e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:05.604925 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411de996-a020-41fb-8afa-b3b922c7812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:05.624894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-411de996-a020-41fb-8afa-b3b922c7812e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:05.626162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-411de996-a020-41fb-8afa-b3b922c7812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:05.639257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d7356-a359-46e7-be10-c5080815bd16 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:05.639546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d7356-a359-46e7-be10-c5080815bd16 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:05.639829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d7356-a359-46e7-be10-c5080815bd16 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:05.641322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 109/210] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:05.646827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6532d05-a055-4908-be29-ab62d11fadd9 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:05.647118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6532d05-a055-4908-be29-ab62d11fadd9 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:05.648180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6532d05-a055-4908-be29-ab62d11fadd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:05.675786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d6532d05-a055-4908-be29-ab62d11fadd9 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:05.677264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d6532d05-a055-4908-be29-ab62d11fadd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:05.728069 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 102/211] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:05.744043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:05.753111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411de996-a020-41fb-8afa-b3b922c7812e None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:05.764664 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 110/212] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:05.789772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6532d05-a055-4908-be29-ab62d11fadd9 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:05.862072 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:05.866178 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411de996-a020-41fb-8afa-b3b922c7812e None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:05.912424 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6532d05-a055-4908-be29-ab62d11fadd9 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:05.914638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdbaa3-cc30-4dea-83f2-353c1860456f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:05.915611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdbaa3-cc30-4dea-83f2-353c1860456f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:05.916662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdbaa3-cc30-4dea-83f2-353c1860456f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:05.920933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdbaa3-cc30-4dea-83f2-353c1860456f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:05.935851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdbaa3-cc30-4dea-83f2-353c1860456f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-57fdbaa3-cc30-4dea-83f2-353c1860456f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:05.946916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:05.947857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:05.948647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:05.948647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:05.948647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:05.948647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:05.948647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:05.948647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:05.948647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:05.948647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:05.948647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:05.948647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:05.948647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:05.948647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:05.949253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 103/213] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:05.954840 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdbaa3-cc30-4dea-83f2-353c1860456f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:05.958354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632dadd-866c-4bb7-83fc-bf053afd7763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:05.960978 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632dadd-866c-4bb7-83fc-bf053afd7763 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:05.961516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632dadd-866c-4bb7-83fc-bf053afd7763 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:05.962010 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632dadd-866c-4bb7-83fc-bf053afd7763 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:05.963426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632dadd-866c-4bb7-83fc-bf053afd7763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:05.965105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632dadd-866c-4bb7-83fc-bf053afd7763 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:05.968230 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 111/214] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:05.973999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632dadd-866c-4bb7-83fc-bf053afd7763 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:06.029473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:06.067677 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdbaa3-cc30-4dea-83f2-353c1860456f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:06.081691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632dadd-866c-4bb7-83fc-bf053afd7763 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:06.103570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750b17d-f28b-4e30-bcc0-ec7d20d74dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:06.104251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750b17d-f28b-4e30-bcc0-ec7d20d74dee None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:06.104542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750b17d-f28b-4e30-bcc0-ec7d20d74dee None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:06.104907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750b17d-f28b-4e30-bcc0-ec7d20d74dee None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:06.105662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750b17d-f28b-4e30-bcc0-ec7d20d74dee None admin] RBAC: Adding query filter params (name=member) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:06.105967 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750b17d-f28b-4e30-bcc0-ec7d20d74dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:06.107348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750b17d-f28b-4e30-bcc0-ec7d20d74dee None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:06.116254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 104/215] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:06.117986 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a2048-2bb7-4559-8b85-c7f298e9b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:06.118374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a864c-7e37-4745-9279-1dbbae0e5863 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:06.118676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a2048-2bb7-4559-8b85-c7f298e9b8c6 None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:06.119017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a2048-2bb7-4559-8b85-c7f298e9b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:06.119256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a864c-7e37-4745-9279-1dbbae0e5863 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:06.119359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a2048-2bb7-4559-8b85-c7f298e9b8c6 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:06.119826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a864c-7e37-4745-9279-1dbbae0e5863 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:06.120054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a2048-2bb7-4559-8b85-c7f298e9b8c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:06.121611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 105/216] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:06.121880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a2048-2bb7-4559-8b85-c7f298e9b8c6 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:06.122798 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a864c-7e37-4745-9279-1dbbae0e5863 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:06.152597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 112/217] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:32:05 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:32:06.152597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f34f91-92bd-41f8-802c-c16d429d23c4 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:06.152597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f34f91-92bd-41f8-802c-c16d429d23c4 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:06.152597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f34f91-92bd-41f8-802c-c16d429d23c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:06.152977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:06.174213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d5f34f91-92bd-41f8-802c-c16d429d23c4 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:06.176210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d5f34f91-92bd-41f8-802c-c16d429d23c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:06.262107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a864c-7e37-4745-9279-1dbbae0e5863 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:06.273618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 113/218] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:06.284074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68831d1-1faf-4019-982e-578081bff62b None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:06.284365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68831d1-1faf-4019-982e-578081bff62b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:06.284698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68831d1-1faf-4019-982e-578081bff62b None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:06.286126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 114/219] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:32:06.291748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85bb40-5bf9-4da6-98f5-729dde21957b None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:06.292038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85bb40-5bf9-4da6-98f5-729dde21957b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:06.292325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85bb40-5bf9-4da6-98f5-729dde21957b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:06.305556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a82f7d-a255-4bae-8680-d8075bd7615a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:06.306840 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a82f7d-a255-4bae-8680-d8075bd7615a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:06.307123 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a82f7d-a255-4bae-8680-d8075bd7615a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:06.307395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a82f7d-a255-4bae-8680-d8075bd7615a None admin] PATH_INFO: `/v3/users/45e7cc4f41324fb39ebe929531a2c6fa` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:06.313986 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-1c85bb40-5bf9-4da6-98f5-729dde21957b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:06.315185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1c85bb40-5bf9-4da6-98f5-729dde21957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:06.323627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a82f7d-a255-4bae-8680-d8075bd7615a None admin] RBAC: Authorizing `identity:get_user(user_id=45e7cc4f41324fb39ebe929531a2c6fa)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:06.325268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a82f7d-a255-4bae-8680-d8075bd7615a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:06.327184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 106/220] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:06 2022] GET /identity/v3/users/45e7cc4f41324fb39ebe929531a2c6fa => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:06.332275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a82f7d-a255-4bae-8680-d8075bd7615a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:06.348597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:06.399412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 115/221] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:06.400840 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85bb40-5bf9-4da6-98f5-729dde21957b None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:06.426188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:06.453908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a82f7d-a255-4bae-8680-d8075bd7615a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:06.498069 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ceba-acdc-405f-ad1e-82b3f989aa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:06.498964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ceba-acdc-405f-ad1e-82b3f989aa3b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:06.499250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ceba-acdc-405f-ad1e-82b3f989aa3b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:06.499523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ceba-acdc-405f-ad1e-82b3f989aa3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:06.511277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2ceba-acdc-405f-ad1e-82b3f989aa3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-a9b2ceba-acdc-405f-ad1e-82b3f989aa3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:06.520086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:06.521486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 107/222] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:32:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:06.522306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ceba-acdc-405f-ad1e-82b3f989aa3b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:06.531619 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85bb40-5bf9-4da6-98f5-729dde21957b None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:06.570674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87406bbe-652d-4c3a-985d-3aac6aab0bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:06.571342 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87406bbe-652d-4c3a-985d-3aac6aab0bc2 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:06.571637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87406bbe-652d-4c3a-985d-3aac6aab0bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:06.571920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87406bbe-652d-4c3a-985d-3aac6aab0bc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:06.609978 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:06.648806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87406bbe-652d-4c3a-985d-3aac6aab0bc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:06.650478 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87406bbe-652d-4c3a-985d-3aac6aab0bc2 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:06.654941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ceba-acdc-405f-ad1e-82b3f989aa3b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:06.696388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a503d-980b-40ab-9659-e7f6f6a32c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:06.697380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a503d-980b-40ab-9659-e7f6f6a32c85 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:06.697778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a503d-980b-40ab-9659-e7f6f6a32c85 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:06.698204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a503d-980b-40ab-9659-e7f6f6a32c85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:06.706883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:06.710475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a503d-980b-40ab-9659-e7f6f6a32c85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-d44a503d-980b-40ab-9659-e7f6f6a32c85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:06.719884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:06.720987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:06.721959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:06.721959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:06.721959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:06.721959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:06.721959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:06.721959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:06.721959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:06.721959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:06.721959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:06.721959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:06.721959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:06.721959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:06.724257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 108/223] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:32:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:06.725349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a503d-980b-40ab-9659-e7f6f6a32c85 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:06.780115 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 116/224] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:06.781503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87406bbe-652d-4c3a-985d-3aac6aab0bc2 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:06.838627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a503d-980b-40ab-9659-e7f6f6a32c85 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:06.873577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:06.878640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951dbe9-b393-464e-b9a9-4ef0ca1c13b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:06.879358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951dbe9-b393-464e-b9a9-4ef0ca1c13b6 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:06.879648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951dbe9-b393-464e-b9a9-4ef0ca1c13b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:06.879930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951dbe9-b393-464e-b9a9-4ef0ca1c13b6 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:06.880743 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951dbe9-b393-464e-b9a9-4ef0ca1c13b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:06.881058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951dbe9-b393-464e-b9a9-4ef0ca1c13b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:06.882987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951dbe9-b393-464e-b9a9-4ef0ca1c13b6 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:06.893963 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 109/225] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:06.894791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951dbe9-b393-464e-b9a9-4ef0ca1c13b6 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:06.913081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87406bbe-652d-4c3a-985d-3aac6aab0bc2 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:06.950975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01cc0d-2a8c-4b0c-82f1-be90f9be9797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:06.951876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01cc0d-2a8c-4b0c-82f1-be90f9be9797 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:06.952349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01cc0d-2a8c-4b0c-82f1-be90f9be9797 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:06.952829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01cc0d-2a8c-4b0c-82f1-be90f9be9797 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:06.953765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01cc0d-2a8c-4b0c-82f1-be90f9be9797 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:06.954209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01cc0d-2a8c-4b0c-82f1-be90f9be9797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:06.955873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01cc0d-2a8c-4b0c-82f1-be90f9be9797 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:06.966644 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 117/226] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:06.967688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01cc0d-2a8c-4b0c-82f1-be90f9be9797 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:06.976132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:07.019220 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951dbe9-b393-464e-b9a9-4ef0ca1c13b6 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:07.057627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99449fb-9edf-455e-b200-076040e54983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:07.058983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99449fb-9edf-455e-b200-076040e54983 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:07.059495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99449fb-9edf-455e-b200-076040e54983 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:07.059999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99449fb-9edf-455e-b200-076040e54983 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:07.061935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99449fb-9edf-455e-b200-076040e54983 None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:07.064772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99449fb-9edf-455e-b200-076040e54983 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:07.066910 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 110/227] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:06 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:07.068021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99449fb-9edf-455e-b200-076040e54983 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:07.091250 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01cc0d-2a8c-4b0c-82f1-be90f9be9797 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:07.120741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:07.126785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8caf9e8-08db-44fb-85e3-202017b31159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:07.128238 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8caf9e8-08db-44fb-85e3-202017b31159 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:07.128829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8caf9e8-08db-44fb-85e3-202017b31159 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:07.128829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8caf9e8-08db-44fb-85e3-202017b31159 None admin] PATH_INFO: `/v3/users/64b59173e29c415d80ea6813088f3f22` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:07.130616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8caf9e8-08db-44fb-85e3-202017b31159 None admin] RBAC: Authorizing `identity:get_user(user_id=64b59173e29c415d80ea6813088f3f22)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:07.132535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8caf9e8-08db-44fb-85e3-202017b31159 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:07.134234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 118/228] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:06 2022] GET /identity/v3/users/64b59173e29c415d80ea6813088f3f22 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:07.135051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8caf9e8-08db-44fb-85e3-202017b31159 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:07.183621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99449fb-9edf-455e-b200-076040e54983 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:07.214348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:07.221632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c7c42-3a95-4fc8-aba1-1a8c29a6eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:07.222830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c7c42-3a95-4fc8-aba1-1a8c29a6eba4 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:07.223314 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c7c42-3a95-4fc8-aba1-1a8c29a6eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:07.223822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c7c42-3a95-4fc8-aba1-1a8c29a6eba4 None admin] PATH_INFO: `/v3/users/64b59173e29c415d80ea6813088f3f22` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:07.225411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c7c42-3a95-4fc8-aba1-1a8c29a6eba4 None admin] RBAC: Authorizing `identity:get_user(user_id=64b59173e29c415d80ea6813088f3f22)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:07.228033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c7c42-3a95-4fc8-aba1-1a8c29a6eba4 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:07.230261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 111/229] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:07 2022] GET /identity/v3/users/64b59173e29c415d80ea6813088f3f22 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:07.231262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c7c42-3a95-4fc8-aba1-1a8c29a6eba4 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:07.242189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8caf9e8-08db-44fb-85e3-202017b31159 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:07.277007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c2ffd-db91-4439-b9c7-26e72e775d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:07.277673 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c2ffd-db91-4439-b9c7-26e72e775d0a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:07.277960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c2ffd-db91-4439-b9c7-26e72e775d0a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:07.278241 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c2ffd-db91-4439-b9c7-26e72e775d0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:07.279096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c2ffd-db91-4439-b9c7-26e72e775d0a None admin] RBAC: Adding query filter params (role.id=82ee29c384454b9aa4db085c9d2e21e8, scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb, user.id=45e7cc4f41324fb39ebe929531a2c6fa) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:07.279413 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c2ffd-db91-4439-b9c7-26e72e775d0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:07.281212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c2ffd-db91-4439-b9c7-26e72e775d0a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:07.286811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 119/230] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:07 2022] GET /identity/v3/role_assignments?user.id=45e7cc4f41324fb39ebe929531a2c6fa&scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb&role.id=82ee29c384454b9aa4db085c9d2e21e8 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:07.290948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c2ffd-db91-4439-b9c7-26e72e775d0a None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:07.351926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c7c42-3a95-4fc8-aba1-1a8c29a6eba4 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:07.362710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:07.394437 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1c393-5e7f-4520-b17f-14c98c5af3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:07.395830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1c393-5e7f-4520-b17f-14c98c5af3ab None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:07.396429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1c393-5e7f-4520-b17f-14c98c5af3ab None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:07.396949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1c393-5e7f-4520-b17f-14c98c5af3ab None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:07.398394 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1c393-5e7f-4520-b17f-14c98c5af3ab None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:07.400221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1c393-5e7f-4520-b17f-14c98c5af3ab None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:07.402400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 112/231] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:07 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:07.408752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c2ffd-db91-4439-b9c7-26e72e775d0a None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:07.414276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1c393-5e7f-4520-b17f-14c98c5af3ab None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:07.473578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bbc87-7f65-4e23-8aa4-3282c5ed62fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:07.474455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bbc87-7f65-4e23-8aa4-3282c5ed62fe None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:07.474743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bbc87-7f65-4e23-8aa4-3282c5ed62fe None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:07.475022 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bbc87-7f65-4e23-8aa4-3282c5ed62fe None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:07.477059 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bbc87-7f65-4e23-8aa4-3282c5ed62fe None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:07.478633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bbc87-7f65-4e23-8aa4-3282c5ed62fe None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:07.480591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 120/232] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:07 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:07.486610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bbc87-7f65-4e23-8aa4-3282c5ed62fe None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:07.498697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:07.534758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1c393-5e7f-4520-b17f-14c98c5af3ab None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:07.571523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4211fcf-1f8e-4566-8355-99a051079fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:07.572162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4211fcf-1f8e-4566-8355-99a051079fbc None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:07.572540 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4211fcf-1f8e-4566-8355-99a051079fbc None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:07.572956 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4211fcf-1f8e-4566-8355-99a051079fbc None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb/users/64b59173e29c415d80ea6813088f3f22/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:07.575063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4211fcf-1f8e-4566-8355-99a051079fbc None admin] RBAC: Authorizing `identity:create_grant(project_id=2a11c903d8ba43eaa621b95be9cb82cb, user_id=64b59173e29c415d80ea6813088f3f22, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:07.576625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4211fcf-1f8e-4566-8355-99a051079fbc None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:07.607388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 113/233] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:32:07 2022] PUT /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb/users/64b59173e29c415d80ea6813088f3f22/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:07.609726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bbc87-7f65-4e23-8aa4-3282c5ed62fe None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:07.659697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:07.666486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320e0f6-78a5-4758-97c9-589eae405097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:07.667461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320e0f6-78a5-4758-97c9-589eae405097 None admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:07.668068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320e0f6-78a5-4758-97c9-589eae405097 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:07.668592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320e0f6-78a5-4758-97c9-589eae405097 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb/users/64b59173e29c415d80ea6813088f3f22/roles/3ce6c4ec0ed142828cb20882e342328e` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:07.673135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320e0f6-78a5-4758-97c9-589eae405097 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a11c903d8ba43eaa621b95be9cb82cb, user_id=64b59173e29c415d80ea6813088f3f22, role_id=3ce6c4ec0ed142828cb20882e342328e)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:07.674647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320e0f6-78a5-4758-97c9-589eae405097 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:07.708788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 121/234] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:32:07 2022] PUT /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb/users/64b59173e29c415d80ea6813088f3f22/roles/3ce6c4ec0ed142828cb20882e342328e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:08.478271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cc2b7-9a92-4c0f-ae8f-b4e5f998ee63 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:08.478808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cc2b7-9a92-4c0f-ae8f-b4e5f998ee63 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:08.478886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cc2b7-9a92-4c0f-ae8f-b4e5f998ee63 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:08.480284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 114/235] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:08.485932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dd289-a1d6-4400-9e69-de3787504a10 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:08.486369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dd289-a1d6-4400-9e69-de3787504a10 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:08.486566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dd289-a1d6-4400-9e69-de3787504a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:08.510390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-ba6dd289-a1d6-4400-9e69-de3787504a10 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:08.511578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ba6dd289-a1d6-4400-9e69-de3787504a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:08.601627 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 122/236] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:08.637165 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92cc2b7-9a92-4c0f-ae8f-b4e5f998ee63 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:08.773831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92cc2b7-9a92-4c0f-ae8f-b4e5f998ee63 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:08.799254 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc2806-17ee-480c-85c6-6f5cc4efa453 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:08.799563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc2806-17ee-480c-85c6-6f5cc4efa453 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:08.799813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc2806-17ee-480c-85c6-6f5cc4efa453 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:08.802535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 123/237] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:08.807129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e46f0-be27-4c0e-af38-407792201a12 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:08.807665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e46f0-be27-4c0e-af38-407792201a12 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:08.808145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e46f0-be27-4c0e-af38-407792201a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:08.814074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:08.827509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fec80-8f0a-4a79-970b-8bc4b3eade5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:08.828318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fec80-8f0a-4a79-970b-8bc4b3eade5d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:08.828655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fec80-8f0a-4a79-970b-8bc4b3eade5d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:08.828896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fec80-8f0a-4a79-970b-8bc4b3eade5d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:08.853402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fec80-8f0a-4a79-970b-8bc4b3eade5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-fc9fec80-8f0a-4a79-970b-8bc4b3eade5d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:08.861403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:08.862284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:08.863052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:08.863052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:08.863052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:08.863052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:08.863052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:08.863052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:08.863052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:08.863052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:08.863052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:08.863052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:08.863052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:08.863052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:08.863660 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-725e46f0-be27-4c0e-af38-407792201a12 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:08.866971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-725e46f0-be27-4c0e-af38-407792201a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:08.868711 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 115/238] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:08.869769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fec80-8f0a-4a79-970b-8bc4b3eade5d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:08.944596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 124/239] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:08.956982 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e46f0-be27-4c0e-af38-407792201a12 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:09.006802 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:09.051437 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fec80-8f0a-4a79-970b-8bc4b3eade5d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:09.089732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf7872-0ee8-4b0f-8f5a-2530f3a5c986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:09.090421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf7872-0ee8-4b0f-8f5a-2530f3a5c986 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:09.090704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf7872-0ee8-4b0f-8f5a-2530f3a5c986 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:09.090979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf7872-0ee8-4b0f-8f5a-2530f3a5c986 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:09.091752 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf7872-0ee8-4b0f-8f5a-2530f3a5c986 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:09.092635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf7872-0ee8-4b0f-8f5a-2530f3a5c986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:09.094089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf7872-0ee8-4b0f-8f5a-2530f3a5c986 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:09.105057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 116/240] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:09.111461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf7872-0ee8-4b0f-8f5a-2530f3a5c986 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:09.119904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e46f0-be27-4c0e-af38-407792201a12 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:09.192584 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f584b1d-d1ac-424c-bdc5-23137ca5bf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:09.193398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f584b1d-d1ac-424c-bdc5-23137ca5bf2c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:09.193686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f584b1d-d1ac-424c-bdc5-23137ca5bf2c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:09.193984 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f584b1d-d1ac-424c-bdc5-23137ca5bf2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:09.195273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f584b1d-d1ac-424c-bdc5-23137ca5bf2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:09.197923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f584b1d-d1ac-424c-bdc5-23137ca5bf2c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:09.199568 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 125/241] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:09.206172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f584b1d-d1ac-424c-bdc5-23137ca5bf2c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:09.211752 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:09.313663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf7872-0ee8-4b0f-8f5a-2530f3a5c986 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:09.381409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9b61c-9dcd-4818-94db-6dff9c93a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:09.382265 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9b61c-9dcd-4818-94db-6dff9c93a02a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:09.382551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9b61c-9dcd-4818-94db-6dff9c93a02a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:09.382828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9b61c-9dcd-4818-94db-6dff9c93a02a None admin] PATH_INFO: `/v3/users/49243286ec014cf99a1e5d14d5dec4f3` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:09.395066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f584b1d-d1ac-424c-bdc5-23137ca5bf2c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:09.407502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9b61c-9dcd-4818-94db-6dff9c93a02a None admin] RBAC: Authorizing `identity:get_user(user_id=49243286ec014cf99a1e5d14d5dec4f3)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:09.409038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9b61c-9dcd-4818-94db-6dff9c93a02a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:09.411107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 117/242] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:09 2022] GET /identity/v3/users/49243286ec014cf99a1e5d14d5dec4f3 => generated 297 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:09.415532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:09.417138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9b61c-9dcd-4818-94db-6dff9c93a02a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:09.433748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a144406-7d41-4844-a792-a8dc29c30833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:09.434434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a144406-7d41-4844-a792-a8dc29c30833 None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:09.435125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a144406-7d41-4844-a792-a8dc29c30833 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:09.435404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a144406-7d41-4844-a792-a8dc29c30833 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:09.436133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a144406-7d41-4844-a792-a8dc29c30833 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:09.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a144406-7d41-4844-a792-a8dc29c30833 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:09.469389 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 126/243] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:32:09 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:32:09.553895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9b61c-9dcd-4818-94db-6dff9c93a02a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:09.562701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:09.594038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931220cd-dd88-4462-9d3c-b4a2e35b6392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:09.594699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931220cd-dd88-4462-9d3c-b4a2e35b6392 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:09.595031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931220cd-dd88-4462-9d3c-b4a2e35b6392 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:09.595308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931220cd-dd88-4462-9d3c-b4a2e35b6392 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:09.670182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931220cd-dd88-4462-9d3c-b4a2e35b6392 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:09.671696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931220cd-dd88-4462-9d3c-b4a2e35b6392 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:09.689739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf572921-9c43-4e3d-b727-c9f2dc60e72c None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:09.690243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf572921-9c43-4e3d-b727-c9f2dc60e72c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:09.690316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf572921-9c43-4e3d-b727-c9f2dc60e72c None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:09.691804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 127/244] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:32:09.697694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27d17f-774b-4218-a2b6-1323e537f09e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:09.697973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27d17f-774b-4218-a2b6-1323e537f09e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:09.698244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27d17f-774b-4218-a2b6-1323e537f09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:09.721902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-8d27d17f-774b-4218-a2b6-1323e537f09e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:09.722509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8d27d17f-774b-4218-a2b6-1323e537f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:09.803772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 128/245] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:09.805902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 118/246] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:09.811653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27d17f-774b-4218-a2b6-1323e537f09e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:09.828241 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931220cd-dd88-4462-9d3c-b4a2e35b6392 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:09.838634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:09.950517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27d17f-774b-4218-a2b6-1323e537f09e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:09.954502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:09.959526 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931220cd-dd88-4462-9d3c-b4a2e35b6392 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:10.007497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eab3a4-1207-4610-9f8b-138d3ea69d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:10.007701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa01111-53a7-4c70-8cf6-f7a9d57eb341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:10.008305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eab3a4-1207-4610-9f8b-138d3ea69d97 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:10.008613 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eab3a4-1207-4610-9f8b-138d3ea69d97 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:10.008901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eab3a4-1207-4610-9f8b-138d3ea69d97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:10.009537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa01111-53a7-4c70-8cf6-f7a9d57eb341 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:10.010018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa01111-53a7-4c70-8cf6-f7a9d57eb341 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:10.010586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa01111-53a7-4c70-8cf6-f7a9d57eb341 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:10.012745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa01111-53a7-4c70-8cf6-f7a9d57eb341 None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:10.014678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa01111-53a7-4c70-8cf6-f7a9d57eb341 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:10.017717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 129/247] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:09 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:10.024120 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa01111-53a7-4c70-8cf6-f7a9d57eb341 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:10.025142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eab3a4-1207-4610-9f8b-138d3ea69d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-c1eab3a4-1207-4610-9f8b-138d3ea69d97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:10.034916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:10.035818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:10.036652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:10.036652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:10.036652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:10.036652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:10.036652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:10.036652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:10.036652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:10.036652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:10.036652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:10.036652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:10.036652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:10.036652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:10.040649 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 119/248] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:10.043107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eab3a4-1207-4610-9f8b-138d3ea69d97 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:10.130687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:10.146920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa01111-53a7-4c70-8cf6-f7a9d57eb341 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:10.172892 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eab3a4-1207-4610-9f8b-138d3ea69d97 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:10.188597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1f186-7c09-404c-b3a3-681fa7b21bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:10.189280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1f186-7c09-404c-b3a3-681fa7b21bdd None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:10.189571 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1f186-7c09-404c-b3a3-681fa7b21bdd None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:10.189851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1f186-7c09-404c-b3a3-681fa7b21bdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:10.190703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1f186-7c09-404c-b3a3-681fa7b21bdd None admin] RBAC: Adding query filter params (role.id=fdcf2d1a66ab4deba04f767592b8c5df, scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb, user.id=49243286ec014cf99a1e5d14d5dec4f3) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:10.191016 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1f186-7c09-404c-b3a3-681fa7b21bdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:10.192470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1f186-7c09-404c-b3a3-681fa7b21bdd None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:10.199258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 130/249] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:10 2022] GET /identity/v3/role_assignments?user.id=49243286ec014cf99a1e5d14d5dec4f3&scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb&role.id=fdcf2d1a66ab4deba04f767592b8c5df => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:10.200289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1fb8d-2d9c-4f55-ad56-db7fa13a4657 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:10.200649 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1fb8d-2d9c-4f55-ad56-db7fa13a4657 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:10.205137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1fb8d-2d9c-4f55-ad56-db7fa13a4657 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:10.206595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 131/250] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:10.216679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b99e7-c559-47e6-96b7-78753c148791 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:10.216978 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b99e7-c559-47e6-96b7-78753c148791 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:10.217235 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b99e7-c559-47e6-96b7-78753c148791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:10.219613 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4323a0e-a913-49a1-ae3a-8704fa0829f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:10.220353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4323a0e-a913-49a1-ae3a-8704fa0829f6 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:10.220683 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4323a0e-a913-49a1-ae3a-8704fa0829f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:10.220974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4323a0e-a913-49a1-ae3a-8704fa0829f6 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:10.221735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4323a0e-a913-49a1-ae3a-8704fa0829f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:10.222044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4323a0e-a913-49a1-ae3a-8704fa0829f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:10.223502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4323a0e-a913-49a1-ae3a-8704fa0829f6 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:10.234942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 120/251] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:10.239648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4323a0e-a913-49a1-ae3a-8704fa0829f6 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:10.253766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-0d2b99e7-c559-47e6-96b7-78753c148791 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:10.255009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0d2b99e7-c559-47e6-96b7-78753c148791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:10.338901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:10.365081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4323a0e-a913-49a1-ae3a-8704fa0829f6 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:10.397915 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:10.408414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569b452-92fc-42d9-ab59-013d352ff442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:10.409340 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569b452-92fc-42d9-ab59-013d352ff442 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:10.409626 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569b452-92fc-42d9-ab59-013d352ff442 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:10.410194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569b452-92fc-42d9-ab59-013d352ff442 None admin] PATH_INFO: `/v3/users/45e7cc4f41324fb39ebe929531a2c6fa` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:10.411598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569b452-92fc-42d9-ab59-013d352ff442 None admin] RBAC: Authorizing `identity:get_user(user_id=45e7cc4f41324fb39ebe929531a2c6fa)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:10.413642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569b452-92fc-42d9-ab59-013d352ff442 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:10.416237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 121/252] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:10 2022] GET /identity/v3/users/45e7cc4f41324fb39ebe929531a2c6fa => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:10.421934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569b452-92fc-42d9-ab59-013d352ff442 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:10.439101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 132/253] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:10.440497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750647b4-d630-4a65-9d11-969a31c9f8c3 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:10.444973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750647b4-d630-4a65-9d11-969a31c9f8c3 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:10.445281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750647b4-d630-4a65-9d11-969a31c9f8c3 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:10.446773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 133/254] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:10.460723 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc46f1-1a11-4c7a-a7a3-14db845e1eb3 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:10.460896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc46f1-1a11-4c7a-a7a3-14db845e1eb3 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:10.461310 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc46f1-1a11-4c7a-a7a3-14db845e1eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:10.487751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-6ccc46f1-1a11-4c7a-a7a3-14db845e1eb3 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:10.488987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6ccc46f1-1a11-4c7a-a7a3-14db845e1eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:10.562940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 134/255] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:10.563819 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc46f1-1a11-4c7a-a7a3-14db845e1eb3 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:10.571323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569b452-92fc-42d9-ab59-013d352ff442 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:10.653268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ff006-ad50-40ea-8148-7f0659425043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:10.654133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ff006-ad50-40ea-8148-7f0659425043 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:10.654415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ff006-ad50-40ea-8148-7f0659425043 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:10.654685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ff006-ad50-40ea-8148-7f0659425043 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:10.656936 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ff006-ad50-40ea-8148-7f0659425043 None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:10.658386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ff006-ad50-40ea-8148-7f0659425043 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:10.663016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 122/256] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:10 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:10.663830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ff006-ad50-40ea-8148-7f0659425043 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:10.734172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:10.744618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc46f1-1a11-4c7a-a7a3-14db845e1eb3 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:10.767888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:10.804912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c2d43-12ce-48d4-8244-0f481217238a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:10.805733 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c2d43-12ce-48d4-8244-0f481217238a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:10.806014 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c2d43-12ce-48d4-8244-0f481217238a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:10.806291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c2d43-12ce-48d4-8244-0f481217238a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:10.810422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ff006-ad50-40ea-8148-7f0659425043 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:10.818273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c2d43-12ce-48d4-8244-0f481217238a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-957c2d43-12ce-48d4-8244-0f481217238a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:10.828863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:10.829763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:10.830546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 135/257] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:32:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:10.831176 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c2d43-12ce-48d4-8244-0f481217238a None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:10.859852 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c5904-9807-48a1-a27d-f6c35f0af569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:10.860748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c5904-9807-48a1-a27d-f6c35f0af569 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:10.861032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c5904-9807-48a1-a27d-f6c35f0af569 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:10.861304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c5904-9807-48a1-a27d-f6c35f0af569 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:10.882493 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c5904-9807-48a1-a27d-f6c35f0af569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-543c5904-9807-48a1-a27d-f6c35f0af569 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:10.899060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:10.901462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 123/258] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:32:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:10.903371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c5904-9807-48a1-a27d-f6c35f0af569 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:11.007982 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c5904-9807-48a1-a27d-f6c35f0af569 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:11.014527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c2d43-12ce-48d4-8244-0f481217238a None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:11.038991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:11.047704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a8d74-fb33-4f1d-91b9-4a0c7f92a9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:11.048380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a8d74-fb33-4f1d-91b9-4a0c7f92a9e6 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:11.048688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a8d74-fb33-4f1d-91b9-4a0c7f92a9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:11.048965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a8d74-fb33-4f1d-91b9-4a0c7f92a9e6 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:11.049779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a8d74-fb33-4f1d-91b9-4a0c7f92a9e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:11.050084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a8d74-fb33-4f1d-91b9-4a0c7f92a9e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:11.052292 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a8d74-fb33-4f1d-91b9-4a0c7f92a9e6 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:11.068664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 124/259] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:11.074648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a8d74-fb33-4f1d-91b9-4a0c7f92a9e6 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:11.088617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343c6f1-2c4f-45ed-b066-70e491e9488f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:11.088617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343c6f1-2c4f-45ed-b066-70e491e9488f None admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:11.088617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343c6f1-2c4f-45ed-b066-70e491e9488f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:11.088617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343c6f1-2c4f-45ed-b066-70e491e9488f None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb/users/45e7cc4f41324fb39ebe929531a2c6fa/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:11.089368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343c6f1-2c4f-45ed-b066-70e491e9488f None admin] RBAC: Authorizing `identity:create_grant(project_id=2a11c903d8ba43eaa621b95be9cb82cb, user_id=45e7cc4f41324fb39ebe929531a2c6fa, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:11.089965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343c6f1-2c4f-45ed-b066-70e491e9488f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:11.119579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 136/260] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:32:10 2022] PUT /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb/users/45e7cc4f41324fb39ebe929531a2c6fa/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:11.120474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343c6f1-2c4f-45ed-b066-70e491e9488f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:11.166064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:11.200670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a8d74-fb33-4f1d-91b9-4a0c7f92a9e6 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:11.249281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9951204-3f4f-4d83-a145-c036e039bf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:11.250350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9951204-3f4f-4d83-a145-c036e039bf22 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:11.250888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9951204-3f4f-4d83-a145-c036e039bf22 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:11.251362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9951204-3f4f-4d83-a145-c036e039bf22 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:11.252335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9951204-3f4f-4d83-a145-c036e039bf22 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:11.257018 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9951204-3f4f-4d83-a145-c036e039bf22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:11.260249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9951204-3f4f-4d83-a145-c036e039bf22 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:11.278501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 125/261] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:11.285690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9951204-3f4f-4d83-a145-c036e039bf22 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:11.346950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343c6f1-2c4f-45ed-b066-70e491e9488f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:11.384251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:11.438449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c3607-bbb0-4bb9-bde3-aeaa56a671a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:11.439354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c3607-bbb0-4bb9-bde3-aeaa56a671a7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:11.439645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c3607-bbb0-4bb9-bde3-aeaa56a671a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:11.439922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c3607-bbb0-4bb9-bde3-aeaa56a671a7 None admin] PATH_INFO: `/v3/users/64b59173e29c415d80ea6813088f3f22` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:11.443514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c3607-bbb0-4bb9-bde3-aeaa56a671a7 None admin] RBAC: Authorizing `identity:get_user(user_id=64b59173e29c415d80ea6813088f3f22)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:11.445081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c3607-bbb0-4bb9-bde3-aeaa56a671a7 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:11.446803 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 137/262] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:11 2022] GET /identity/v3/users/64b59173e29c415d80ea6813088f3f22 => generated 283 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:11.455138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c3607-bbb0-4bb9-bde3-aeaa56a671a7 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:11.490886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9951204-3f4f-4d83-a145-c036e039bf22 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:11.541649 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a71a0-3459-40a0-ab44-0993245ebe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:11.542797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a71a0-3459-40a0-ab44-0993245ebe92 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:11.543081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a71a0-3459-40a0-ab44-0993245ebe92 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:11.543767 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a71a0-3459-40a0-ab44-0993245ebe92 None admin] PATH_INFO: `/v3/users/64b59173e29c415d80ea6813088f3f22` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:11.545385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a71a0-3459-40a0-ab44-0993245ebe92 None admin] RBAC: Authorizing `identity:get_user(user_id=64b59173e29c415d80ea6813088f3f22)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:11.547091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a71a0-3459-40a0-ab44-0993245ebe92 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:11.550043 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 126/263] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:11 2022] GET /identity/v3/users/64b59173e29c415d80ea6813088f3f22 => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:11.558673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a71a0-3459-40a0-ab44-0993245ebe92 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:11.590789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:11.640428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c3607-bbb0-4bb9-bde3-aeaa56a671a7 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:11.672702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a71a0-3459-40a0-ab44-0993245ebe92 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:11.689997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:11.713368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a865c-de38-4991-94b6-8e45bcbba473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:11.714043 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a865c-de38-4991-94b6-8e45bcbba473 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:11.714335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a865c-de38-4991-94b6-8e45bcbba473 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:11.714617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a865c-de38-4991-94b6-8e45bcbba473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:11.729273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261c602-8517-414e-ad19-60d378a4c6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:11.729940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261c602-8517-414e-ad19-60d378a4c6fc None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:11.730227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261c602-8517-414e-ad19-60d378a4c6fc None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:11.730507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261c602-8517-414e-ad19-60d378a4c6fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:11.789090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a865c-de38-4991-94b6-8e45bcbba473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:11.790764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a865c-de38-4991-94b6-8e45bcbba473 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:11.848228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261c602-8517-414e-ad19-60d378a4c6fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:11.849658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261c602-8517-414e-ad19-60d378a4c6fc None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:11.893543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 127/264] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:11.903832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a865c-de38-4991-94b6-8e45bcbba473 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:11.932915 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:12.012274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:12.047570 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 138/265] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:12.053580 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261c602-8517-414e-ad19-60d378a4c6fc None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:12.092194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a865c-de38-4991-94b6-8e45bcbba473 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:12.156029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e776d6-ac8c-447d-826b-ba38db84f775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:12.160689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e776d6-ac8c-447d-826b-ba38db84f775 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:12.160689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e776d6-ac8c-447d-826b-ba38db84f775 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:12.160689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e776d6-ac8c-447d-826b-ba38db84f775 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:12.161771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e776d6-ac8c-447d-826b-ba38db84f775 None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:12.163643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e776d6-ac8c-447d-826b-ba38db84f775 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:12.166079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 128/266] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:11 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:12.172709 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e776d6-ac8c-447d-826b-ba38db84f775 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:12.234483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261c602-8517-414e-ad19-60d378a4c6fc None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:12.296558 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47075d-19cf-4050-8705-7aae5898ea06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:12.297478 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47075d-19cf-4050-8705-7aae5898ea06 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:12.297767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47075d-19cf-4050-8705-7aae5898ea06 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:12.298047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47075d-19cf-4050-8705-7aae5898ea06 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:12.299270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47075d-19cf-4050-8705-7aae5898ea06 None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:12.306449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47075d-19cf-4050-8705-7aae5898ea06 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:12.308810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 139/267] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:12 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:12.314805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47075d-19cf-4050-8705-7aae5898ea06 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:12.363364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:12.407794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e776d6-ac8c-447d-826b-ba38db84f775 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:12.409521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:12.522571 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f92ba-5296-4334-800a-27f325ff2a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:12.523246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f92ba-5296-4334-800a-27f325ff2a91 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:12.523529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f92ba-5296-4334-800a-27f325ff2a91 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:12.523806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f92ba-5296-4334-800a-27f325ff2a91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:12.524671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f92ba-5296-4334-800a-27f325ff2a91 None admin] RBAC: Adding query filter params (role.id=3ce6c4ec0ed142828cb20882e342328e, scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb, user.id=64b59173e29c415d80ea6813088f3f22) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:12.524984 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f92ba-5296-4334-800a-27f325ff2a91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:12.526553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f92ba-5296-4334-800a-27f325ff2a91 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:12.528041 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47075d-19cf-4050-8705-7aae5898ea06 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:12.542122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 129/268] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:12 2022] GET /identity/v3/role_assignments?user.id=64b59173e29c415d80ea6813088f3f22&scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb&role.id=3ce6c4ec0ed142828cb20882e342328e => generated 608 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:12.588348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b6d2b-6b51-4426-b2c8-462344389e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:12.589056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b6d2b-6b51-4426-b2c8-462344389e29 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:12.589341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b6d2b-6b51-4426-b2c8-462344389e29 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:12.589620 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b6d2b-6b51-4426-b2c8-462344389e29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:12.590471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b6d2b-6b51-4426-b2c8-462344389e29 None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb, user.id=64b59173e29c415d80ea6813088f3f22) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:12.590782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b6d2b-6b51-4426-b2c8-462344389e29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:12.592227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b6d2b-6b51-4426-b2c8-462344389e29 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:12.599036 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 140/269] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:12 2022] GET /identity/v3/role_assignments?user.id=64b59173e29c415d80ea6813088f3f22&scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:12.711754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7d293-635b-490c-8610-361f97720cf1 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:12.712254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7d293-635b-490c-8610-361f97720cf1 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:12.712254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7d293-635b-490c-8610-361f97720cf1 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:12.714095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 130/270] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:12.720145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5adcf-f165-4ff1-84e3-8f25d6bf45fc None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:12.720443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5adcf-f165-4ff1-84e3-8f25d6bf45fc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:12.720741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5adcf-f165-4ff1-84e3-8f25d6bf45fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:12.745477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-56e5adcf-f165-4ff1-84e3-8f25d6bf45fc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:12.746721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-56e5adcf-f165-4ff1-84e3-8f25d6bf45fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:12.822963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 141/271] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:12.861616 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7d293-635b-490c-8610-361f97720cf1 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:12.956319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be0396-f773-45f0-bb44-1cd94684264e None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:12.956853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be0396-f773-45f0-bb44-1cd94684264e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:12.957110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be0396-f773-45f0-bb44-1cd94684264e None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:12.958630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 142/272] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:12.964722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e08219-042b-42c4-82d9-152277614877 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:12.964948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e08219-042b-42c4-82d9-152277614877 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:12.965229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e08219-042b-42c4-82d9-152277614877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:12.979361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7d293-635b-490c-8610-361f97720cf1 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:12.995504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-77e08219-042b-42c4-82d9-152277614877 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:12.996729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-77e08219-042b-42c4-82d9-152277614877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:13.026841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f297d3c-5b7a-4010-9992-31bccb9dcfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:13.028001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f297d3c-5b7a-4010-9992-31bccb9dcfe3 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:13.028507 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f297d3c-5b7a-4010-9992-31bccb9dcfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:13.029033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f297d3c-5b7a-4010-9992-31bccb9dcfe3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:13.041738 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f297d3c-5b7a-4010-9992-31bccb9dcfe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-1f297d3c-5b7a-4010-9992-31bccb9dcfe3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:13.049946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:13.050855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:13.051642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:13.051642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:13.051642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:13.051642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:13.051642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:13.051642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:13.051642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:13.051642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:13.051642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:13.051642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:13.051642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:13.051642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:13.055905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 131/273] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:13.055905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018cbc9-acca-4723-a874-757dc409f84e None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:13.055905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018cbc9-acca-4723-a874-757dc409f84e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:13.056177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018cbc9-acca-4723-a874-757dc409f84e None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:13.057997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 132/274] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:13.061433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018cbc9-acca-4723-a874-757dc409f84e None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:13.065621 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:13.078823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:13.105661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 143/275] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:13.107023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e887eb3-2c90-406d-993f-1b4f99e32529 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:13.107310 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e887eb3-2c90-406d-993f-1b4f99e32529 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:13.107629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e887eb3-2c90-406d-993f-1b4f99e32529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:13.125764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-2e887eb3-2c90-406d-993f-1b4f99e32529 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:13.126952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2e887eb3-2c90-406d-993f-1b4f99e32529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:13.181442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018cbc9-acca-4723-a874-757dc409f84e None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:13.194362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 144/276] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:13.195020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e887eb3-2c90-406d-993f-1b4f99e32529 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:13.233635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336ec0e-3b6e-49e4-929b-cf73edf654c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:13.234376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336ec0e-3b6e-49e4-929b-cf73edf654c1 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:13.234660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336ec0e-3b6e-49e4-929b-cf73edf654c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:13.234940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336ec0e-3b6e-49e4-929b-cf73edf654c1 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:13.235676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336ec0e-3b6e-49e4-929b-cf73edf654c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:13.235979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336ec0e-3b6e-49e4-929b-cf73edf654c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:13.237397 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336ec0e-3b6e-49e4-929b-cf73edf654c1 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:13.246565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 133/277] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:13.246565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336ec0e-3b6e-49e4-929b-cf73edf654c1 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:13.308692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e887eb3-2c90-406d-993f-1b4f99e32529 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:13.316966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:13.319548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:13.352514 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336ec0e-3b6e-49e4-929b-cf73edf654c1 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:13.353358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5fa64-330c-4cdb-8866-ab61d98c56b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:13.354410 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5fa64-330c-4cdb-8866-ab61d98c56b3 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:13.355496 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5fa64-330c-4cdb-8866-ab61d98c56b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:13.355785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5fa64-330c-4cdb-8866-ab61d98c56b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:13.356981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5fa64-330c-4cdb-8866-ab61d98c56b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:13.358437 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5fa64-330c-4cdb-8866-ab61d98c56b3 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:13.365068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 145/278] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:13.365068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5fa64-330c-4cdb-8866-ab61d98c56b3 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:13.386930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac0e75-5f05-4605-af7d-17111e450773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:13.387727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac0e75-5f05-4605-af7d-17111e450773 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:13.388004 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac0e75-5f05-4605-af7d-17111e450773 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:13.388279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac0e75-5f05-4605-af7d-17111e450773 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:13.399204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac0e75-5f05-4605-af7d-17111e450773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-b7ac0e75-5f05-4605-af7d-17111e450773 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:13.407446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:13.408293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:13.409115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 134/279] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:13.409757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac0e75-5f05-4605-af7d-17111e450773 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:13.488690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5fa64-330c-4cdb-8866-ab61d98c56b3 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:13.524387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac0e75-5f05-4605-af7d-17111e450773 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:13.524594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094014eb-924c-4949-887e-83f368a753c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:13.525435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094014eb-924c-4949-887e-83f368a753c5 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:13.525726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094014eb-924c-4949-887e-83f368a753c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:13.526158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094014eb-924c-4949-887e-83f368a753c5 None admin] PATH_INFO: `/v3/users/49243286ec014cf99a1e5d14d5dec4f3` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:13.528085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094014eb-924c-4949-887e-83f368a753c5 None admin] RBAC: Authorizing `identity:get_user(user_id=49243286ec014cf99a1e5d14d5dec4f3)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:13.529602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094014eb-924c-4949-887e-83f368a753c5 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:13.531721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 146/280] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:13 2022] GET /identity/v3/users/49243286ec014cf99a1e5d14d5dec4f3 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:13.532290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094014eb-924c-4949-887e-83f368a753c5 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:13.557358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:13.564185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cb185-04ce-423e-ac31-406e370999bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:13.564947 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cb185-04ce-423e-ac31-406e370999bd None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:13.565235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cb185-04ce-423e-ac31-406e370999bd None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:13.565659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cb185-04ce-423e-ac31-406e370999bd None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:13.566224 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cb185-04ce-423e-ac31-406e370999bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:13.567665 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cb185-04ce-423e-ac31-406e370999bd None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:13.579204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:13.593240 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 135/281] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:32:13 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:32:13.594073 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cb185-04ce-423e-ac31-406e370999bd None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:13.663258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094014eb-924c-4949-887e-83f368a753c5 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:13.705083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864094c8-89a8-4d9f-8b62-66869ba3c219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:13.706304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864094c8-89a8-4d9f-8b62-66869ba3c219 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:13.706785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864094c8-89a8-4d9f-8b62-66869ba3c219 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:13.707275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864094c8-89a8-4d9f-8b62-66869ba3c219 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:13.708235 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864094c8-89a8-4d9f-8b62-66869ba3c219 None admin] RBAC: Adding query filter params (name=member) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:13.708814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864094c8-89a8-4d9f-8b62-66869ba3c219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:13.710764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864094c8-89a8-4d9f-8b62-66869ba3c219 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:13.719769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cb185-04ce-423e-ac31-406e370999bd None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:13.722027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 147/282] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:13.732713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864094c8-89a8-4d9f-8b62-66869ba3c219 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:13.759710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32756932-0fd6-40e8-b7b3-53c1caca8b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:13.760513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32756932-0fd6-40e8-b7b3-53c1caca8b5f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:13.760944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32756932-0fd6-40e8-b7b3-53c1caca8b5f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:13.761224 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32756932-0fd6-40e8-b7b3-53c1caca8b5f None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:13.762436 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32756932-0fd6-40e8-b7b3-53c1caca8b5f None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:13.763891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32756932-0fd6-40e8-b7b3-53c1caca8b5f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:13.766008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 136/283] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:13 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:13.772667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32756932-0fd6-40e8-b7b3-53c1caca8b5f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:13.844795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:13.869908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864094c8-89a8-4d9f-8b62-66869ba3c219 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:13.874050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:13.899002 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32756932-0fd6-40e8-b7b3-53c1caca8b5f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:13.922188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a698d-8612-44f9-a974-68f2101b77e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:13.923398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a698d-8612-44f9-a974-68f2101b77e7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:13.923956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a698d-8612-44f9-a974-68f2101b77e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:13.924416 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a698d-8612-44f9-a974-68f2101b77e7 None admin] PATH_INFO: `/v3/users/6c9bfb72093340ce90fb56b40f479391` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:13.942857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a698d-8612-44f9-a974-68f2101b77e7 None admin] RBAC: Authorizing `identity:get_user(user_id=6c9bfb72093340ce90fb56b40f479391)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:13.944680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc7790-e34e-4bab-9101-f5b31da80464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:13.945605 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a698d-8612-44f9-a974-68f2101b77e7 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:13.945734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc7790-e34e-4bab-9101-f5b31da80464 None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:13.945910 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc7790-e34e-4bab-9101-f5b31da80464 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:13.946154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc7790-e34e-4bab-9101-f5b31da80464 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb/users/49243286ec014cf99a1e5d14d5dec4f3/roles/fdcf2d1a66ab4deba04f767592b8c5df` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:13.947616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 148/284] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:13 2022] GET /identity/v3/users/6c9bfb72093340ce90fb56b40f479391 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:13.948374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc7790-e34e-4bab-9101-f5b31da80464 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a11c903d8ba43eaa621b95be9cb82cb, user_id=49243286ec014cf99a1e5d14d5dec4f3, role_id=fdcf2d1a66ab4deba04f767592b8c5df)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:13.949899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc7790-e34e-4bab-9101-f5b31da80464 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:13.952990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a698d-8612-44f9-a974-68f2101b77e7 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:13.973379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 137/285] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:32:13 2022] PUT /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb/users/49243286ec014cf99a1e5d14d5dec4f3/roles/fdcf2d1a66ab4deba04f767592b8c5df => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:32:14.016687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3ae23-3c0b-4369-bac2-5263b680ca07 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:14.016687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3ae23-3c0b-4369-bac2-5263b680ca07 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:14.016687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3ae23-3c0b-4369-bac2-5263b680ca07 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:14.016687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 138/286] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:32:14.021164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311a7e6-a2e6-44e4-a001-229ec7f90e71 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:14.021462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311a7e6-a2e6-44e4-a001-229ec7f90e71 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:14.021771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311a7e6-a2e6-44e4-a001-229ec7f90e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:14.058963 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5311a7e6-a2e6-44e4-a001-229ec7f90e71 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:14.060275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5311a7e6-a2e6-44e4-a001-229ec7f90e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:14.097316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a698d-8612-44f9-a974-68f2101b77e7 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:14.139641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a245473-9bcb-44ff-9ff1-5c156af3976a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:14.140329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245473-9bcb-44ff-9ff1-5c156af3976a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:14.140634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245473-9bcb-44ff-9ff1-5c156af3976a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:14.140917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245473-9bcb-44ff-9ff1-5c156af3976a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:14.141777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a245473-9bcb-44ff-9ff1-5c156af3976a None admin] RBAC: Adding query filter params (role.id=82ee29c384454b9aa4db085c9d2e21e8, user.id=6c9bfb72093340ce90fb56b40f479391, scope.system=all) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:14.142091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a245473-9bcb-44ff-9ff1-5c156af3976a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:14.143865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a245473-9bcb-44ff-9ff1-5c156af3976a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:14.150668 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 149/287] 10.222.0.52 () {56 vars in 1308 bytes} [Mon Jul 25 08:32:13 2022] GET /identity/v3/role_assignments?user.id=6c9bfb72093340ce90fb56b40f479391&scope.system=all&role.id=82ee29c384454b9aa4db085c9d2e21e8 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:14.163802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 139/288] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:14.219428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a245473-9bcb-44ff-9ff1-5c156af3976a None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:14.251904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:14.359176 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a245473-9bcb-44ff-9ff1-5c156af3976a None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:14.430214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b3fd7-117a-4181-af35-3221b2d2eef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:14.431031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b3fd7-117a-4181-af35-3221b2d2eef7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:14.431315 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b3fd7-117a-4181-af35-3221b2d2eef7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:14.431588 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b3fd7-117a-4181-af35-3221b2d2eef7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:14.462450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b3fd7-117a-4181-af35-3221b2d2eef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-292b3fd7-117a-4181-af35-3221b2d2eef7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:14.470387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:14.471276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:14.472067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 150/289] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:14.481710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311a7e6-a2e6-44e4-a001-229ec7f90e71 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:14.594694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:14.680492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311a7e6-a2e6-44e4-a001-229ec7f90e71 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:14.717122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a859ab8-8540-4591-8714-a968bf40f4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:14.717802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a859ab8-8540-4591-8714-a968bf40f4e5 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:14.718084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a859ab8-8540-4591-8714-a968bf40f4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:14.718360 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a859ab8-8540-4591-8714-a968bf40f4e5 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:14.719221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a859ab8-8540-4591-8714-a968bf40f4e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:14.719527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a859ab8-8540-4591-8714-a968bf40f4e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:14.720933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a859ab8-8540-4591-8714-a968bf40f4e5 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:14.734682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 140/290] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:14.736002 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b3fd7-117a-4181-af35-3221b2d2eef7 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:14.846650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:14.867002 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b3fd7-117a-4181-af35-3221b2d2eef7 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:14.920068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8d4eb-dc5b-4d02-b6ab-d20948f22d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:14.920995 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8d4eb-dc5b-4d02-b6ab-d20948f22d25 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:14.921283 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8d4eb-dc5b-4d02-b6ab-d20948f22d25 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:14.921560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8d4eb-dc5b-4d02-b6ab-d20948f22d25 None admin] PATH_INFO: `/v3/users/45e7cc4f41324fb39ebe929531a2c6fa` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:14.922852 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8d4eb-dc5b-4d02-b6ab-d20948f22d25 None admin] RBAC: Authorizing `identity:get_user(user_id=45e7cc4f41324fb39ebe929531a2c6fa)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:14.924420 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8d4eb-dc5b-4d02-b6ab-d20948f22d25 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:14.930290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 151/291] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:14 2022] GET /identity/v3/users/45e7cc4f41324fb39ebe929531a2c6fa => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:14.935903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a859ab8-8540-4591-8714-a968bf40f4e5 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:15.071223 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a859ab8-8540-4591-8714-a968bf40f4e5 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:15.078328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:15.114192 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa94b4-0ed3-4ad7-b35d-55aa3f9bb317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:15.114772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa94b4-0ed3-4ad7-b35d-55aa3f9bb317 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:15.115056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa94b4-0ed3-4ad7-b35d-55aa3f9bb317 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:15.115331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa94b4-0ed3-4ad7-b35d-55aa3f9bb317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:15.204493 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa94b4-0ed3-4ad7-b35d-55aa3f9bb317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:15.205973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa94b4-0ed3-4ad7-b35d-55aa3f9bb317 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:15.352390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 141/292] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:15.357850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8d4eb-dc5b-4d02-b6ab-d20948f22d25 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:15.486498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8d4eb-dc5b-4d02-b6ab-d20948f22d25 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:15.523497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b09ca5-abb8-4d0d-9bf1-1105c75065c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:15.524372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b09ca5-abb8-4d0d-9bf1-1105c75065c4 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:15.524681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b09ca5-abb8-4d0d-9bf1-1105c75065c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:15.524957 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b09ca5-abb8-4d0d-9bf1-1105c75065c4 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:15.526191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b09ca5-abb8-4d0d-9bf1-1105c75065c4 None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:15.527638 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b09ca5-abb8-4d0d-9bf1-1105c75065c4 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:15.529378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 152/293] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:15 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:15.535645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa94b4-0ed3-4ad7-b35d-55aa3f9bb317 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:15.544757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:15.656207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa94b4-0ed3-4ad7-b35d-55aa3f9bb317 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:15.706708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29b092-de27-4004-8cb9-bbe049cec2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:15.707454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29b092-de27-4004-8cb9-bbe049cec2f2 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:15.707742 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29b092-de27-4004-8cb9-bbe049cec2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:15.708105 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29b092-de27-4004-8cb9-bbe049cec2f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:15.709029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29b092-de27-4004-8cb9-bbe049cec2f2 None admin] RBAC: Adding query filter params (role.id=82ee29c384454b9aa4db085c9d2e21e8, scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb, user.id=45e7cc4f41324fb39ebe929531a2c6fa) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:15.709371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29b092-de27-4004-8cb9-bbe049cec2f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:15.710805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29b092-de27-4004-8cb9-bbe049cec2f2 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:15.721251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 142/294] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:15 2022] GET /identity/v3/role_assignments?user.id=45e7cc4f41324fb39ebe929531a2c6fa&scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb&role.id=82ee29c384454b9aa4db085c9d2e21e8 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:16.335577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23b9b7-6fd1-47e2-b072-d566be399219 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:16.335963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23b9b7-6fd1-47e2-b072-d566be399219 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:16.336153 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23b9b7-6fd1-47e2-b072-d566be399219 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:16.337808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 153/295] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:16.344130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99294e-8987-4865-bae1-dcd9f0cf3c91 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:16.344508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99294e-8987-4865-bae1-dcd9f0cf3c91 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:16.344823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99294e-8987-4865-bae1-dcd9f0cf3c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:16.368863 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3a99294e-8987-4865-bae1-dcd9f0cf3c91 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:16.370381 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3a99294e-8987-4865-bae1-dcd9f0cf3c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:16.394549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b954-e361-4e54-93f3-56bf28989435 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:16.395054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b954-e361-4e54-93f3-56bf28989435 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:16.395054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b954-e361-4e54-93f3-56bf28989435 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:16.399114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 154/296] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:16.402835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92afe81-c906-4f90-98e8-1c9067569651 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:16.403130 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92afe81-c906-4f90-98e8-1c9067569651 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:16.403412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92afe81-c906-4f90-98e8-1c9067569651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:16.424951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-e92afe81-c906-4f90-98e8-1c9067569651 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:16.426314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e92afe81-c906-4f90-98e8-1c9067569651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:16.445421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:16.456620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:16.462441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 143/297] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:16.466213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f8fa6-ad84-4ce6-beaa-7d5547c8dab7 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:16.467625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f8fa6-ad84-4ce6-beaa-7d5547c8dab7 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:16.468106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f8fa6-ad84-4ce6-beaa-7d5547c8dab7 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:16.469762 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 144/298] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:16.478578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348e091-2b19-4faf-893a-15af209ff8af None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:16.479249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348e091-2b19-4faf-893a-15af209ff8af None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:16.479809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348e091-2b19-4faf-893a-15af209ff8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:16.498846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-7348e091-2b19-4faf-893a-15af209ff8af None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:16.500025 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7348e091-2b19-4faf-893a-15af209ff8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:16.502311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 155/299] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:16.509577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92afe81-c906-4f90-98e8-1c9067569651 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:16.570058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 145/300] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:16.570757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348e091-2b19-4faf-893a-15af209ff8af None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:16.631592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92afe81-c906-4f90-98e8-1c9067569651 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:16.666286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533a3e2-5b17-4d2e-a100-b4f68fcd01ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:16.667086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a3e2-5b17-4d2e-a100-b4f68fcd01ba None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:16.667370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a3e2-5b17-4d2e-a100-b4f68fcd01ba None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:16.667645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a3e2-5b17-4d2e-a100-b4f68fcd01ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:16.668863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533a3e2-5b17-4d2e-a100-b4f68fcd01ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:16.670372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533a3e2-5b17-4d2e-a100-b4f68fcd01ba None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:16.672041 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 156/301] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:16.673504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533a3e2-5b17-4d2e-a100-b4f68fcd01ba None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:16.685071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348e091-2b19-4faf-893a-15af209ff8af None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:16.696953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:16.718550 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:16.725056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4beab3-a354-4c35-9c89-056ca4488a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:16.725860 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4beab3-a354-4c35-9c89-056ca4488a43 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:16.726143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4beab3-a354-4c35-9c89-056ca4488a43 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:16.726445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4beab3-a354-4c35-9c89-056ca4488a43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:16.736849 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4beab3-a354-4c35-9c89-056ca4488a43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-8b4beab3-a354-4c35-9c89-056ca4488a43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:16.745033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:16.745909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:16.746691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 146/302] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:16.747337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4beab3-a354-4c35-9c89-056ca4488a43 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:16.791269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533a3e2-5b17-4d2e-a100-b4f68fcd01ba None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:16.831191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13206515-b75c-49e4-878f-daf62cd73897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:16.831926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13206515-b75c-49e4-878f-daf62cd73897 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:16.832287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13206515-b75c-49e4-878f-daf62cd73897 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:16.832601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13206515-b75c-49e4-878f-daf62cd73897 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:16.842930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13206515-b75c-49e4-878f-daf62cd73897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-13206515-b75c-49e4-878f-daf62cd73897 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:16.850791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:16.851661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:16.852439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 157/303] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:16.853118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13206515-b75c-49e4-878f-daf62cd73897 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:16.858445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4beab3-a354-4c35-9c89-056ca4488a43 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:16.891500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165cca1-391d-4faf-906b-fe190eb3774b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:16.892156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165cca1-391d-4faf-906b-fe190eb3774b None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:16.892438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165cca1-391d-4faf-906b-fe190eb3774b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:16.892749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165cca1-391d-4faf-906b-fe190eb3774b None admin] PATH_INFO: `/v3/groups` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:16.893434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165cca1-391d-4faf-906b-fe190eb3774b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:16.894885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165cca1-391d-4faf-906b-fe190eb3774b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:16.925773 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 147/304] 10.222.0.52 () {60 vars in 1134 bytes} [Mon Jul 25 08:32:16 2022] POST /identity/v3/groups => generated 229 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:32:16.925773 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de4d0d-acc1-4a81-8e78-b58c316882f9 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:16.925773 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de4d0d-acc1-4a81-8e78-b58c316882f9 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:16.925773 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de4d0d-acc1-4a81-8e78-b58c316882f9 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:16.925773 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 148/305] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:16.925773 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de4d0d-acc1-4a81-8e78-b58c316882f9 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:16.943050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:16.969435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13206515-b75c-49e4-878f-daf62cd73897 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:16.986142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:17.010202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568acd4-0bae-46cc-88c6-1e6b8a3e1bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:17.012138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568acd4-0bae-46cc-88c6-1e6b8a3e1bd0 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:17.012449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568acd4-0bae-46cc-88c6-1e6b8a3e1bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:17.012778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568acd4-0bae-46cc-88c6-1e6b8a3e1bd0 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:17.013559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568acd4-0bae-46cc-88c6-1e6b8a3e1bd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:17.013876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568acd4-0bae-46cc-88c6-1e6b8a3e1bd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:17.015311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568acd4-0bae-46cc-88c6-1e6b8a3e1bd0 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:17.026750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 158/306] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:17.031915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e5e01-0fe0-4138-a4b3-33d19d27b219 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:17.031915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e5e01-0fe0-4138-a4b3-33d19d27b219 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:17.031915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e5e01-0fe0-4138-a4b3-33d19d27b219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:17.049903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-717e5e01-0fe0-4138-a4b3-33d19d27b219 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:17.051149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-717e5e01-0fe0-4138-a4b3-33d19d27b219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:17.052616 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de4d0d-acc1-4a81-8e78-b58c316882f9 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:17.095919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0ff75-5f74-409f-835f-443c12174fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:17.096642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0ff75-5f74-409f-835f-443c12174fd6 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:17.096926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0ff75-5f74-409f-835f-443c12174fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:17.097200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0ff75-5f74-409f-835f-443c12174fd6 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:17.097962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0ff75-5f74-409f-835f-443c12174fd6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:17.098266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0ff75-5f74-409f-835f-443c12174fd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:17.099674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0ff75-5f74-409f-835f-443c12174fd6 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:17.110114 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 149/307] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:17.112927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0ff75-5f74-409f-835f-443c12174fd6 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:17.132808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 159/308] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:17.133907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e5e01-0fe0-4138-a4b3-33d19d27b219 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:17.211064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:17.222436 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0ff75-5f74-409f-835f-443c12174fd6 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:17.235829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:17.242404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e5e01-0fe0-4138-a4b3-33d19d27b219 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:17.259462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26914384-a407-4ca3-8ca5-85fa1e827e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:17.260336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26914384-a407-4ca3-8ca5-85fa1e827e8a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:17.260644 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26914384-a407-4ca3-8ca5-85fa1e827e8a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:17.260926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26914384-a407-4ca3-8ca5-85fa1e827e8a None admin] PATH_INFO: `/v3/users/6c9bfb72093340ce90fb56b40f479391` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:17.262110 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26914384-a407-4ca3-8ca5-85fa1e827e8a None admin] RBAC: Authorizing `identity:get_user(user_id=6c9bfb72093340ce90fb56b40f479391)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:17.263550 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26914384-a407-4ca3-8ca5-85fa1e827e8a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:17.265220 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 150/309] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:17 2022] GET /identity/v3/users/6c9bfb72093340ce90fb56b40f479391 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:17.270557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26914384-a407-4ca3-8ca5-85fa1e827e8a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:17.276657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2ab1b-5a4f-44ab-aaff-17d1d7df31f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:17.277544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2ab1b-5a4f-44ab-aaff-17d1d7df31f3 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:17.277829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2ab1b-5a4f-44ab-aaff-17d1d7df31f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:17.278103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2ab1b-5a4f-44ab-aaff-17d1d7df31f3 None admin] PATH_INFO: `/v3/users/49243286ec014cf99a1e5d14d5dec4f3` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:17.279309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2ab1b-5a4f-44ab-aaff-17d1d7df31f3 None admin] RBAC: Authorizing `identity:get_user(user_id=49243286ec014cf99a1e5d14d5dec4f3)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:17.281050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2ab1b-5a4f-44ab-aaff-17d1d7df31f3 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:17.287009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 160/310] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:17 2022] GET /identity/v3/users/49243286ec014cf99a1e5d14d5dec4f3 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:17.287009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2ab1b-5a4f-44ab-aaff-17d1d7df31f3 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:17.389261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26914384-a407-4ca3-8ca5-85fa1e827e8a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:17.394560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2ab1b-5a4f-44ab-aaff-17d1d7df31f3 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:17.425937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9219bc-560e-433c-9d60-1940011e9a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:17.426759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9219bc-560e-433c-9d60-1940011e9a73 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:17.427043 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9219bc-560e-433c-9d60-1940011e9a73 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:17.427318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9219bc-560e-433c-9d60-1940011e9a73 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:17.431228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247a4ef-4456-4049-b6cd-8388b5263d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:17.432162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247a4ef-4456-4049-b6cd-8388b5263d04 None admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:17.432367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247a4ef-4456-4049-b6cd-8388b5263d04 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:17.432671 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247a4ef-4456-4049-b6cd-8388b5263d04 None admin] PATH_INFO: `/v3/system/users/6c9bfb72093340ce90fb56b40f479391/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:17.434163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247a4ef-4456-4049-b6cd-8388b5263d04 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6c9bfb72093340ce90fb56b40f479391, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:17.435538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247a4ef-4456-4049-b6cd-8388b5263d04 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:17.437574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9219bc-560e-433c-9d60-1940011e9a73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:17.444493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 161/311] 10.222.0.52 () {58 vars in 1334 bytes} [Mon Jul 25 08:32:17 2022] PUT /identity/v3/system/users/6c9bfb72093340ce90fb56b40f479391/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:17.445780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247a4ef-4456-4049-b6cd-8388b5263d04 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-3f9219bc-560e-433c-9d60-1940011e9a73 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:17.446079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:17.446951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:17.447777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:17.447777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:17.447777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:17.447777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:17.447777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:17.447777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:17.447777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:17.447777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:17.447777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:17.447777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:17.447777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:17.447777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:17.452242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 151/312] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:17.453292 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9219bc-560e-433c-9d60-1940011e9a73 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:17.454817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:17.510699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:17.576562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247a4ef-4456-4049-b6cd-8388b5263d04 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:17.599299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9219bc-560e-433c-9d60-1940011e9a73 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:17.629947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6c17c-7944-4fb9-9d4b-92ef5f03c50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:17.630645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6c17c-7944-4fb9-9d4b-92ef5f03c50b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:17.630928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6c17c-7944-4fb9-9d4b-92ef5f03c50b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:17.631252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6c17c-7944-4fb9-9d4b-92ef5f03c50b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:17.650051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893e48e-878a-4d8d-ab5d-a56306620eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:17.651199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893e48e-878a-4d8d-ab5d-a56306620eb3 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:17.651636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893e48e-878a-4d8d-ab5d-a56306620eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:17.652335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893e48e-878a-4d8d-ab5d-a56306620eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:17.653680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893e48e-878a-4d8d-ab5d-a56306620eb3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:17.654142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893e48e-878a-4d8d-ab5d-a56306620eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:17.655948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893e48e-878a-4d8d-ab5d-a56306620eb3 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:17.669182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 152/313] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:17.675109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893e48e-878a-4d8d-ab5d-a56306620eb3 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:17.724414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6c17c-7944-4fb9-9d4b-92ef5f03c50b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:17.726770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6c17c-7944-4fb9-9d4b-92ef5f03c50b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:17.730266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:17.784659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893e48e-878a-4d8d-ab5d-a56306620eb3 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:17.787095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:17.824131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf33486-8a0d-4f45-b671-60905816d4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:17.825393 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf33486-8a0d-4f45-b671-60905816d4eb None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:17.825874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf33486-8a0d-4f45-b671-60905816d4eb None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:17.826340 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf33486-8a0d-4f45-b671-60905816d4eb None admin] PATH_INFO: `/v3/users/1c24431a118f4506aa9e64972bc31f00` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:17.841765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf33486-8a0d-4f45-b671-60905816d4eb None admin] RBAC: Authorizing `identity:get_user(user_id=1c24431a118f4506aa9e64972bc31f00)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:17.843484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf33486-8a0d-4f45-b671-60905816d4eb None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:17.850462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 153/314] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:17 2022] GET /identity/v3/users/1c24431a118f4506aa9e64972bc31f00 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:17.851612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf33486-8a0d-4f45-b671-60905816d4eb None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:17.863087 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 162/315] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:17.865579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6c17c-7944-4fb9-9d4b-92ef5f03c50b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:17.973958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6c17c-7944-4fb9-9d4b-92ef5f03c50b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:17.977524 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:17.980466 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf33486-8a0d-4f45-b671-60905816d4eb None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:18.023255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1017c-b9ce-49e8-998a-2044cfe6a126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:18.024755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1017c-b9ce-49e8-998a-2044cfe6a126 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:18.025296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1017c-b9ce-49e8-998a-2044cfe6a126 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:18.025777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1017c-b9ce-49e8-998a-2044cfe6a126 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:18.025878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3f0b8-2137-4a4b-8aeb-bb2be3689344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:18.026486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3f0b8-2137-4a4b-8aeb-bb2be3689344 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:18.026784 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3f0b8-2137-4a4b-8aeb-bb2be3689344 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:18.027055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3f0b8-2137-4a4b-8aeb-bb2be3689344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:18.027895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3f0b8-2137-4a4b-8aeb-bb2be3689344 None admin] RBAC: Adding query filter params (role.id=fdcf2d1a66ab4deba04f767592b8c5df, user.id=1c24431a118f4506aa9e64972bc31f00, scope.system=all) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:18.027999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1017c-b9ce-49e8-998a-2044cfe6a126 None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:18.028204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3f0b8-2137-4a4b-8aeb-bb2be3689344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:18.029662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3f0b8-2137-4a4b-8aeb-bb2be3689344 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:18.030307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1017c-b9ce-49e8-998a-2044cfe6a126 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:18.037743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 163/316] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:17 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:18.038748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1017c-b9ce-49e8-998a-2044cfe6a126 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:18.041230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 154/317] 10.222.0.52 () {56 vars in 1308 bytes} [Mon Jul 25 08:32:17 2022] GET /identity/v3/role_assignments?user.id=1c24431a118f4506aa9e64972bc31f00&scope.system=all&role.id=fdcf2d1a66ab4deba04f767592b8c5df => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:18.152944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1017c-b9ce-49e8-998a-2044cfe6a126 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:18.190924 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8253ed-964f-45a4-afaf-a9667eaf7d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:18.191845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8253ed-964f-45a4-afaf-a9667eaf7d05 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:18.192271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8253ed-964f-45a4-afaf-a9667eaf7d05 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:18.192723 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8253ed-964f-45a4-afaf-a9667eaf7d05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:18.194276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8253ed-964f-45a4-afaf-a9667eaf7d05 None admin] RBAC: Adding query filter params (role.id=fdcf2d1a66ab4deba04f767592b8c5df, scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb, user.id=49243286ec014cf99a1e5d14d5dec4f3) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:18.194756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8253ed-964f-45a4-afaf-a9667eaf7d05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:18.196354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8253ed-964f-45a4-afaf-a9667eaf7d05 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:18.207196 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 164/318] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:18 2022] GET /identity/v3/role_assignments?user.id=49243286ec014cf99a1e5d14d5dec4f3&scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb&role.id=fdcf2d1a66ab4deba04f767592b8c5df => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:18.952351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4c3e6-8548-430b-a739-01f693c17bd1 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:18.952920 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4c3e6-8548-430b-a739-01f693c17bd1 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:18.952997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4c3e6-8548-430b-a739-01f693c17bd1 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:18.954369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 155/319] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:18.959857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a91f-e7b5-49d5-8cc0-22be25d334dd None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:18.960225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a91f-e7b5-49d5-8cc0-22be25d334dd None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:18.960465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a91f-e7b5-49d5-8cc0-22be25d334dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:18.979151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-da45a91f-e7b5-49d5-8cc0-22be25d334dd None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:18.980351 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-da45a91f-e7b5-49d5-8cc0-22be25d334dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:19.009052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:19.050493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 165/320] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:19.071843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4c3e6-8548-430b-a739-01f693c17bd1 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:19.097351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:19.178643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4c3e6-8548-430b-a739-01f693c17bd1 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:19.212516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696a089-55d1-4e1e-96ed-062063526d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:19.213269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696a089-55d1-4e1e-96ed-062063526d45 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:19.213557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696a089-55d1-4e1e-96ed-062063526d45 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:19.213836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696a089-55d1-4e1e-96ed-062063526d45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:19.215028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696a089-55d1-4e1e-96ed-062063526d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:19.216482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696a089-55d1-4e1e-96ed-062063526d45 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:19.218308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 156/321] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:19.223632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a91f-e7b5-49d5-8cc0-22be25d334dd None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:19.329837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a91f-e7b5-49d5-8cc0-22be25d334dd None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:19.363874 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d46080-27ff-4189-8d88-dcae78938eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:19.364556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d46080-27ff-4189-8d88-dcae78938eb6 None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:19.364870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d46080-27ff-4189-8d88-dcae78938eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:19.365147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d46080-27ff-4189-8d88-dcae78938eb6 None admin] PATH_INFO: `/v3/groups` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:19.365878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d46080-27ff-4189-8d88-dcae78938eb6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:19.367318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d46080-27ff-4189-8d88-dcae78938eb6 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:19.385217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 166/322] 10.222.0.52 () {60 vars in 1134 bytes} [Mon Jul 25 08:32:19 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:32:19.472231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ab5a4-978a-4042-84b3-752c56fc82e8 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:19.472540 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ab5a4-978a-4042-84b3-752c56fc82e8 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:19.472752 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ab5a4-978a-4042-84b3-752c56fc82e8 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:19.474302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 157/323] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:19.480177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592933ab-56b0-41d2-9cec-af39881ac995 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:19.480470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592933ab-56b0-41d2-9cec-af39881ac995 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:19.480789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592933ab-56b0-41d2-9cec-af39881ac995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:19.500328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-592933ab-56b0-41d2-9cec-af39881ac995 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:19.501571 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-592933ab-56b0-41d2-9cec-af39881ac995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:19.524326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:19.575923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 167/324] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:19.598959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ab5a4-978a-4042-84b3-752c56fc82e8 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:19.702962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:19.713198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ab5a4-978a-4042-84b3-752c56fc82e8 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:19.756990 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a9cfd-6298-49b4-92fa-b611b9295254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:19.757785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a9cfd-6298-49b4-92fa-b611b9295254 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:19.758076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a9cfd-6298-49b4-92fa-b611b9295254 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:19.758360 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a9cfd-6298-49b4-92fa-b611b9295254 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:19.769588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a9cfd-6298-49b4-92fa-b611b9295254 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-449a9cfd-6298-49b4-92fa-b611b9295254 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:19.777548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:19.778443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:19.779239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:19.779239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:19.779239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:19.779239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:19.779239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:19.779239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:19.779239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:19.779239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:19.779239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:19.779239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:19.779239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:19.779239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:19.783058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 158/325] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:19.785664 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592933ab-56b0-41d2-9cec-af39881ac995 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:19.908178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592933ab-56b0-41d2-9cec-af39881ac995 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:19.947887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb7cb8-0b3f-42e2-8cd1-0ba7f63fae0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:19.948945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb7cb8-0b3f-42e2-8cd1-0ba7f63fae0d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:19.949439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb7cb8-0b3f-42e2-8cd1-0ba7f63fae0d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:19.949943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb7cb8-0b3f-42e2-8cd1-0ba7f63fae0d None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:19.950922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb7cb8-0b3f-42e2-8cd1-0ba7f63fae0d None admin] RBAC: Adding query filter params (name=member) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:19.951434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb7cb8-0b3f-42e2-8cd1-0ba7f63fae0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:19.953074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb7cb8-0b3f-42e2-8cd1-0ba7f63fae0d None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:19.962776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 168/326] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:19.968095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a9cfd-6298-49b4-92fa-b611b9295254 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:20.058247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b9ac0-347d-4435-b3b8-0b862671fb46 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:20.058852 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b9ac0-347d-4435-b3b8-0b862671fb46 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:20.059392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b9ac0-347d-4435-b3b8-0b862671fb46 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:20.061021 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 169/327] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:32:20.074013 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff87943-a74a-4e7a-ae38-07885d9a6251 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:20.074693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff87943-a74a-4e7a-ae38-07885d9a6251 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:20.075177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff87943-a74a-4e7a-ae38-07885d9a6251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:20.094897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a9cfd-6298-49b4-92fa-b611b9295254 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:20.110096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-5ff87943-a74a-4e7a-ae38-07885d9a6251 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:20.110096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-5ff87943-a74a-4e7a-ae38-07885d9a6251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:20.130580 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:20.153417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375e1d0-3ecc-4af9-804a-8c7119e6dfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:20.154310 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375e1d0-3ecc-4af9-804a-8c7119e6dfc9 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:20.154839 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375e1d0-3ecc-4af9-804a-8c7119e6dfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:20.155120 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375e1d0-3ecc-4af9-804a-8c7119e6dfc9 None admin] PATH_INFO: `/v3/users/6c9bfb72093340ce90fb56b40f479391` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:20.156545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375e1d0-3ecc-4af9-804a-8c7119e6dfc9 None admin] RBAC: Authorizing `identity:get_user(user_id=6c9bfb72093340ce90fb56b40f479391)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:20.158042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375e1d0-3ecc-4af9-804a-8c7119e6dfc9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:20.159833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 159/328] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:19 2022] GET /identity/v3/users/6c9bfb72093340ce90fb56b40f479391 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:20.171422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375e1d0-3ecc-4af9-804a-8c7119e6dfc9 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:20.196970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:20.227081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 170/329] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:20.263043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff87943-a74a-4e7a-ae38-07885d9a6251 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:20.282763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375e1d0-3ecc-4af9-804a-8c7119e6dfc9 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:20.319822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb6266-49fe-417b-98b5-1e3952cf7fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:20.320543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb6266-49fe-417b-98b5-1e3952cf7fc0 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:20.320854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb6266-49fe-417b-98b5-1e3952cf7fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:20.321140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb6266-49fe-417b-98b5-1e3952cf7fc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:20.321987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb6266-49fe-417b-98b5-1e3952cf7fc0 None admin] RBAC: Adding query filter params (role.id=82ee29c384454b9aa4db085c9d2e21e8, user.id=6c9bfb72093340ce90fb56b40f479391, scope.system=all) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:20.322302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb6266-49fe-417b-98b5-1e3952cf7fc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:20.323774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb6266-49fe-417b-98b5-1e3952cf7fc0 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:20.336518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 160/330] 10.222.0.52 () {56 vars in 1308 bytes} [Mon Jul 25 08:32:20 2022] GET /identity/v3/role_assignments?user.id=6c9bfb72093340ce90fb56b40f479391&scope.system=all&role.id=82ee29c384454b9aa4db085c9d2e21e8 => generated 510 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:20.474398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff87943-a74a-4e7a-ae38-07885d9a6251 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:20.526624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38419f28-5cfb-436b-831b-f4759b534afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:20.527782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38419f28-5cfb-436b-831b-f4759b534afc None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:20.528299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38419f28-5cfb-436b-831b-f4759b534afc None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:20.528820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38419f28-5cfb-436b-831b-f4759b534afc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:20.541676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38419f28-5cfb-436b-831b-f4759b534afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-38419f28-5cfb-436b-831b-f4759b534afc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:20.550033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:20.550927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:20.551726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:20.551726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:20.551726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:20.551726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:20.551726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:20.551726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:20.551726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:20.551726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:20.551726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:20.551726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:20.551726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:20.551726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:20.556155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 171/331] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:20.562240 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb6266-49fe-417b-98b5-1e3952cf7fc0 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:20.661091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:20.687078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb6266-49fe-417b-98b5-1e3952cf7fc0 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:20.727631 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a5c66-f763-469e-90a0-4086073cd62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:20.728326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a5c66-f763-469e-90a0-4086073cd62b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:20.728726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a5c66-f763-469e-90a0-4086073cd62b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:20.729015 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a5c66-f763-469e-90a0-4086073cd62b None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:20.729772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a5c66-f763-469e-90a0-4086073cd62b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:20.730078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a5c66-f763-469e-90a0-4086073cd62b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:20.731458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a5c66-f763-469e-90a0-4086073cd62b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:20.740296 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 161/332] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:20.746200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38419f28-5cfb-436b-831b-f4759b534afc None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:20.763592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:20.888808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38419f28-5cfb-436b-831b-f4759b534afc None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:20.932653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407aaf64-fba1-4c25-90f6-9f48990da210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:20.933544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407aaf64-fba1-4c25-90f6-9f48990da210 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:20.933826 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407aaf64-fba1-4c25-90f6-9f48990da210 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:20.934103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407aaf64-fba1-4c25-90f6-9f48990da210 None admin] PATH_INFO: `/v3/users/1c24431a118f4506aa9e64972bc31f00` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:20.941333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407aaf64-fba1-4c25-90f6-9f48990da210 None admin] RBAC: Authorizing `identity:get_user(user_id=1c24431a118f4506aa9e64972bc31f00)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:20.942798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407aaf64-fba1-4c25-90f6-9f48990da210 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:20.945171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 172/333] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:20 2022] GET /identity/v3/users/1c24431a118f4506aa9e64972bc31f00 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:20.950994 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a5c66-f763-469e-90a0-4086073cd62b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:21.080833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a5c66-f763-469e-90a0-4086073cd62b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:21.115251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ad4af-79e6-4808-bf7c-fac14b3af8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:21.116132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ad4af-79e6-4808-bf7c-fac14b3af8af None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:21.116408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ad4af-79e6-4808-bf7c-fac14b3af8af None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:21.116712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ad4af-79e6-4808-bf7c-fac14b3af8af None admin] PATH_INFO: `/v3/system/users/1c24431a118f4506aa9e64972bc31f00/roles/fdcf2d1a66ab4deba04f767592b8c5df` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:21.118221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ad4af-79e6-4808-bf7c-fac14b3af8af None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1c24431a118f4506aa9e64972bc31f00, role_id=fdcf2d1a66ab4deba04f767592b8c5df)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:21.119604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ad4af-79e6-4808-bf7c-fac14b3af8af None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:21.142846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 162/334] 10.222.0.52 () {58 vars in 1334 bytes} [Mon Jul 25 08:32:20 2022] PUT /identity/v3/system/users/1c24431a118f4506aa9e64972bc31f00/roles/fdcf2d1a66ab4deba04f767592b8c5df => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:22.312696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0f182-cca4-4b3e-ad2d-ec87962e6a4a None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:22.313118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0f182-cca4-4b3e-ad2d-ec87962e6a4a None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:22.313246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0f182-cca4-4b3e-ad2d-ec87962e6a4a None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:22.315164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 173/335] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:22.320105 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337cd81-ba0f-4e4d-abbe-9f61ee922375 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:22.320518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337cd81-ba0f-4e4d-abbe-9f61ee922375 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:22.320702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337cd81-ba0f-4e4d-abbe-9f61ee922375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:22.328161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:22.343452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-2337cd81-ba0f-4e4d-abbe-9f61ee922375 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:22.344673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2337cd81-ba0f-4e4d-abbe-9f61ee922375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:22.426046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 163/336] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:22.458346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0f182-cca4-4b3e-ad2d-ec87962e6a4a None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:22.480270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d4179-527e-4ad8-b6d9-ef7e71c27291 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:22.480904 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d4179-527e-4ad8-b6d9-ef7e71c27291 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:22.481414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d4179-527e-4ad8-b6d9-ef7e71c27291 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:22.483578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 164/337] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:22.491331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a0c6b-bc68-481c-bd61-e376f7ac5282 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:22.491882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a0c6b-bc68-481c-bd61-e376f7ac5282 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:22.492375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a0c6b-bc68-481c-bd61-e376f7ac5282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:22.512411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-374a0c6b-bc68-481c-bd61-e376f7ac5282 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:22.513978 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-374a0c6b-bc68-481c-bd61-e376f7ac5282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:22.596136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 165/338] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:22.598326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e865cd9-78eb-41c3-aca5-919246103ca8 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:22.599096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e865cd9-78eb-41c3-aca5-919246103ca8 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:22.600104 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e865cd9-78eb-41c3-aca5-919246103ca8 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:22.601285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:22.605935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 166/339] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:22.611499 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2c29a-1fc4-4d5b-8d75-4a3f109b1946 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:22.612055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2c29a-1fc4-4d5b-8d75-4a3f109b1946 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:22.612530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2c29a-1fc4-4d5b-8d75-4a3f109b1946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:22.640401 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0f182-cca4-4b3e-ad2d-ec87962e6a4a None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:22.641620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-46c2c29a-1fc4-4d5b-8d75-4a3f109b1946 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:22.642999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-46c2c29a-1fc4-4d5b-8d75-4a3f109b1946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:22.684835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8285157-4eae-478b-8bf1-7ac28c8f94c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:22.686054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8285157-4eae-478b-8bf1-7ac28c8f94c7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:22.686595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8285157-4eae-478b-8bf1-7ac28c8f94c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:22.687137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8285157-4eae-478b-8bf1-7ac28c8f94c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:22.700515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8285157-4eae-478b-8bf1-7ac28c8f94c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-e8285157-4eae-478b-8bf1-7ac28c8f94c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:22.709766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:22.710653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:22.711461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:22.711461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:22.711461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:22.711461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:22.711461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:22.711461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:22.711461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:22.711461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:22.711461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:22.711461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:22.711461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:22.711461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:22.714202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 174/340] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:32:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:22.714369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:22.715328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8285157-4eae-478b-8bf1-7ac28c8f94c7 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:22.726288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 167/341] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:22.727407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2c29a-1fc4-4d5b-8d75-4a3f109b1946 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:22.845233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8285157-4eae-478b-8bf1-7ac28c8f94c7 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:22.846624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2c29a-1fc4-4d5b-8d75-4a3f109b1946 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:22.890516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7a7fb-637c-4bd3-8c1d-024f43350fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:22.891200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7a7fb-637c-4bd3-8c1d-024f43350fb9 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:22.891596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7a7fb-637c-4bd3-8c1d-024f43350fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:22.891973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7a7fb-637c-4bd3-8c1d-024f43350fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:22.896805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7a7fb-637c-4bd3-8c1d-024f43350fb9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:22.896805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7a7fb-637c-4bd3-8c1d-024f43350fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:22.896805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7a7fb-637c-4bd3-8c1d-024f43350fb9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:22.900509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff80329-5654-4da0-adbd-0090f801218d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:22.901556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff80329-5654-4da0-adbd-0090f801218d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:22.902018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff80329-5654-4da0-adbd-0090f801218d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:22.902470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff80329-5654-4da0-adbd-0090f801218d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:22.904233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 168/342] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:22.909525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:22.909664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7a7fb-637c-4bd3-8c1d-024f43350fb9 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:22.920385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff80329-5654-4da0-adbd-0090f801218d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-eff80329-5654-4da0-adbd-0090f801218d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:22.932647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:22.933556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:22.934329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:22.934329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:22.934329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:22.934329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:22.934329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:22.934329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:22.934329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:22.934329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:22.934329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:22.934329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:22.934329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:22.934329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:22.937018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 175/343] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:22.938018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff80329-5654-4da0-adbd-0090f801218d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:22.970032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:23.019116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7a7fb-637c-4bd3-8c1d-024f43350fb9 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:23.064495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97bfe5-08a2-4bc6-b8e0-20468320bb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:23.065333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97bfe5-08a2-4bc6-b8e0-20468320bb0d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:23.065620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97bfe5-08a2-4bc6-b8e0-20468320bb0d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:23.065898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97bfe5-08a2-4bc6-b8e0-20468320bb0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:23.090619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97bfe5-08a2-4bc6-b8e0-20468320bb0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:23.102927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff80329-5654-4da0-adbd-0090f801218d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-1a97bfe5-08a2-4bc6-b8e0-20468320bb0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:23.105406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:23.106304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:23.107116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 169/344] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:23.108544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97bfe5-08a2-4bc6-b8e0-20468320bb0d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:23.172320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acd165-a469-422b-bc4b-1e2f2d808cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:23.173037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acd165-a469-422b-bc4b-1e2f2d808cd0 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:23.173326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acd165-a469-422b-bc4b-1e2f2d808cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:23.173608 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acd165-a469-422b-bc4b-1e2f2d808cd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:23.273385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97bfe5-08a2-4bc6-b8e0-20468320bb0d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:23.291016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:23.314384 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d6ea2-a76c-45b6-bfd8-f182ac23c340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:23.315062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d6ea2-a76c-45b6-bfd8-f182ac23c340 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:23.315345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d6ea2-a76c-45b6-bfd8-f182ac23c340 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:23.315619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d6ea2-a76c-45b6-bfd8-f182ac23c340 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:23.316388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d6ea2-a76c-45b6-bfd8-f182ac23c340 None admin] RBAC: Adding query filter params (name=member) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:23.316708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d6ea2-a76c-45b6-bfd8-f182ac23c340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:23.319043 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d6ea2-a76c-45b6-bfd8-f182ac23c340 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:23.325961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:23.330096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 170/345] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:23.331123 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d6ea2-a76c-45b6-bfd8-f182ac23c340 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:23.349685 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acd165-a469-422b-bc4b-1e2f2d808cd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:23.351185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acd165-a469-422b-bc4b-1e2f2d808cd0 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:23.440828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d6ea2-a76c-45b6-bfd8-f182ac23c340 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:23.465744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 176/346] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:23.466624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acd165-a469-422b-bc4b-1e2f2d808cd0 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:23.484101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94dbbb-b3ce-49ee-af92-a814568b4c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:23.484809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94dbbb-b3ce-49ee-af92-a814568b4c30 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:23.485096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94dbbb-b3ce-49ee-af92-a814568b4c30 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:23.485373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94dbbb-b3ce-49ee-af92-a814568b4c30 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:23.486132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94dbbb-b3ce-49ee-af92-a814568b4c30 None admin] RBAC: Adding query filter params (name=member) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:23.486442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94dbbb-b3ce-49ee-af92-a814568b4c30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:23.487820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94dbbb-b3ce-49ee-af92-a814568b4c30 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:23.509075 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 171/347] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:23.509989 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94dbbb-b3ce-49ee-af92-a814568b4c30 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:23.571873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:23.572768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:23.578199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acd165-a469-422b-bc4b-1e2f2d808cd0 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:23.617219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7124d25-f44b-4b9f-9788-5ed516ef7bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:23.617886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7124d25-f44b-4b9f-9788-5ed516ef7bbf None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:23.618177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7124d25-f44b-4b9f-9788-5ed516ef7bbf None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:23.618462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7124d25-f44b-4b9f-9788-5ed516ef7bbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:23.664760 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94dbbb-b3ce-49ee-af92-a814568b4c30 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:23.702696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc725d3-bdde-4f28-8b96-023e27864ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:23.703487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc725d3-bdde-4f28-8b96-023e27864ad5 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:23.704052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc725d3-bdde-4f28-8b96-023e27864ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:23.704329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc725d3-bdde-4f28-8b96-023e27864ad5 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:23.706205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc725d3-bdde-4f28-8b96-023e27864ad5 None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:23.707296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7124d25-f44b-4b9f-9788-5ed516ef7bbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:23.707930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc725d3-bdde-4f28-8b96-023e27864ad5 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:23.708831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7124d25-f44b-4b9f-9788-5ed516ef7bbf None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:23.710058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 172/348] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:23 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:23.711098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc725d3-bdde-4f28-8b96-023e27864ad5 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:23.824009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:23.835821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 177/349] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:23.837527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e3a5e-721d-4510-b6cb-efaf17aad2fd None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:23.837899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e3a5e-721d-4510-b6cb-efaf17aad2fd None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:23.838259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e3a5e-721d-4510-b6cb-efaf17aad2fd None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:23.839937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 178/350] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:23.840685 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e3a5e-721d-4510-b6cb-efaf17aad2fd None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:23.843595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc725d3-bdde-4f28-8b96-023e27864ad5 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:23.855699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:23.890283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8eace-1bd8-41a7-8d1e-0658d6c67a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:23.890945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8eace-1bd8-41a7-8d1e-0658d6c67a3b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:23.891229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8eace-1bd8-41a7-8d1e-0658d6c67a3b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:23.891505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8eace-1bd8-41a7-8d1e-0658d6c67a3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:23.967840 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8eace-1bd8-41a7-8d1e-0658d6c67a3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:23.969352 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8eace-1bd8-41a7-8d1e-0658d6c67a3b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:23.974998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e3a5e-721d-4510-b6cb-efaf17aad2fd None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:24.019745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b033e-b41e-48a0-a92d-64c42fdaf5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:24.020946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b033e-b41e-48a0-a92d-64c42fdaf5de None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.021471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b033e-b41e-48a0-a92d-64c42fdaf5de None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.021894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b033e-b41e-48a0-a92d-64c42fdaf5de None admin] PATH_INFO: `/v3/groups/cf32feb4691047099d7e3ea98965d01f` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.031463 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b033e-b41e-48a0-a92d-64c42fdaf5de None admin] RBAC: Authorizing `identity:get_group(group_id=cf32feb4691047099d7e3ea98965d01f)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:24.033114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b033e-b41e-48a0-a92d-64c42fdaf5de None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:24.035781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 179/351] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:23 2022] GET /identity/v3/groups/cf32feb4691047099d7e3ea98965d01f => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:24.036664 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b033e-b41e-48a0-a92d-64c42fdaf5de None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:24.090121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 173/352] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:24.091507 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5774355-0508-4eff-8da8-bcac6846edec None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.091802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5774355-0508-4eff-8da8-bcac6846edec None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.092087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5774355-0508-4eff-8da8-bcac6846edec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.111201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-c5774355-0508-4eff-8da8-bcac6846edec None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:24.112455 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c5774355-0508-4eff-8da8-bcac6846edec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:24.115053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:24.119057 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:24.173127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b033e-b41e-48a0-a92d-64c42fdaf5de None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:24.197701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 174/353] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:24.198558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5774355-0508-4eff-8da8-bcac6846edec None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:24.223836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76959b67-5134-4c84-81c2-0dd5d130f2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:24.225227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76959b67-5134-4c84-81c2-0dd5d130f2b8 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.225512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76959b67-5134-4c84-81c2-0dd5d130f2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.226054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76959b67-5134-4c84-81c2-0dd5d130f2b8 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.227305 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76959b67-5134-4c84-81c2-0dd5d130f2b8 None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:24.229008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76959b67-5134-4c84-81c2-0dd5d130f2b8 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:24.231003 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 180/354] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:24 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:24.231659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76959b67-5134-4c84-81c2-0dd5d130f2b8 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:24.330505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5774355-0508-4eff-8da8-bcac6846edec None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:24.352892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76959b67-5134-4c84-81c2-0dd5d130f2b8 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:24.371249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72491fd2-59bf-428a-8c3a-7d509d98596a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:24.372894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72491fd2-59bf-428a-8c3a-7d509d98596a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.372894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72491fd2-59bf-428a-8c3a-7d509d98596a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.372894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72491fd2-59bf-428a-8c3a-7d509d98596a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.374377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72491fd2-59bf-428a-8c3a-7d509d98596a None admin] RBAC: Adding query filter params (group.id=cf32feb4691047099d7e3ea98965d01f, role.id=3ce6c4ec0ed142828cb20882e342328e, scope.project.id=d68d17929cf4421f977087a339234c2a) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:24.374697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72491fd2-59bf-428a-8c3a-7d509d98596a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:24.377584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72491fd2-59bf-428a-8c3a-7d509d98596a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:24.381988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:24.383911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 175/355] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Jul 25 08:32:24 2022] GET /identity/v3/role_assignments?group.id=cf32feb4691047099d7e3ea98965d01f&scope.project.id=d68d17929cf4421f977087a339234c2a&role.id=3ce6c4ec0ed142828cb20882e342328e => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:24.385261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204195a-e431-4130-8510-a85c8b18bc8e None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.385557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204195a-e431-4130-8510-a85c8b18bc8e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.385842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204195a-e431-4130-8510-a85c8b18bc8e None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.387170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 176/356] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:24.387906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204195a-e431-4130-8510-a85c8b18bc8e None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:24.390962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:24.400619 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f09477-8132-4862-a96f-639829e3ccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:24.401411 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f09477-8132-4862-a96f-639829e3ccee None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.402593 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f09477-8132-4862-a96f-639829e3ccee None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.402894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f09477-8132-4862-a96f-639829e3ccee None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.404876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f09477-8132-4862-a96f-639829e3ccee None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:24.406558 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f09477-8132-4862-a96f-639829e3ccee None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:24.413374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 181/357] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:24 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:24.413374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f09477-8132-4862-a96f-639829e3ccee None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:24.541480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204195a-e431-4130-8510-a85c8b18bc8e None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:24.553908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f09477-8132-4862-a96f-639829e3ccee None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:24.591385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524eee9-bb73-4abc-a823-02cc6836ca89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:24.592207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524eee9-bb73-4abc-a823-02cc6836ca89 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.592497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524eee9-bb73-4abc-a823-02cc6836ca89 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.592828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524eee9-bb73-4abc-a823-02cc6836ca89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.599989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18743ae-83fa-4fb2-8edf-4d90fe7f2b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:24.601543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18743ae-83fa-4fb2-8edf-4d90fe7f2b1f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.601974 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18743ae-83fa-4fb2-8edf-4d90fe7f2b1f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.602393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18743ae-83fa-4fb2-8edf-4d90fe7f2b1f None admin] PATH_INFO: `/v3/groups/cf32feb4691047099d7e3ea98965d01f` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.603962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524eee9-bb73-4abc-a823-02cc6836ca89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:24.604386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18743ae-83fa-4fb2-8edf-4d90fe7f2b1f None admin] RBAC: Authorizing `identity:get_group(group_id=cf32feb4691047099d7e3ea98965d01f)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:24.605938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18743ae-83fa-4fb2-8edf-4d90fe7f2b1f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:24.607796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 182/358] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:24 2022] GET /identity/v3/groups/cf32feb4691047099d7e3ea98965d01f => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:24.609658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41047162-a342-4774-bda8-d82b3c47f9db None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.609948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41047162-a342-4774-bda8-d82b3c47f9db None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.610223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41047162-a342-4774-bda8-d82b3c47f9db None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.611729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 183/359] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-0524eee9-bb73-4abc-a823-02cc6836ca89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:24.612780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:24.613676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:24.614484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:24.614484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:24.614484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:24.614484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:24.614484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:24.614484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:24.614484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:24.614484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:24.614484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:24.614484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:24.614484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:24.614484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:24.615131 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988931f8-fecf-47bb-a223-d8527fa15c98 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.615131 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988931f8-fecf-47bb-a223-d8527fa15c98 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.615131 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988931f8-fecf-47bb-a223-d8527fa15c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.615377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 177/360] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:32:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:24.615377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524eee9-bb73-4abc-a823-02cc6836ca89 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:24.641671 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-988931f8-fecf-47bb-a223-d8527fa15c98 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:24.642883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-988931f8-fecf-47bb-a223-d8527fa15c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:24.700483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:24.720974 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:24.731595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 184/361] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:24.733170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988931f8-fecf-47bb-a223-d8527fa15c98 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:24.751803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524eee9-bb73-4abc-a823-02cc6836ca89 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:24.799622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4a4a7-6680-4eb5-ae57-df1283a9d781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:24.800622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4a4a7-6680-4eb5-ae57-df1283a9d781 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.800911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4a4a7-6680-4eb5-ae57-df1283a9d781 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.801188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4a4a7-6680-4eb5-ae57-df1283a9d781 None admin] PATH_INFO: `/v3/groups/cf32feb4691047099d7e3ea98965d01f` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.802432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4a4a7-6680-4eb5-ae57-df1283a9d781 None admin] RBAC: Authorizing `identity:get_group(group_id=cf32feb4691047099d7e3ea98965d01f)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:24.803894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4a4a7-6680-4eb5-ae57-df1283a9d781 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:24.805868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 178/362] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:24 2022] GET /identity/v3/groups/cf32feb4691047099d7e3ea98965d01f => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:24.811644 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850cbdd-1162-4fc0-a945-0a7f95002d92 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.811907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850cbdd-1162-4fc0-a945-0a7f95002d92 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.812192 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850cbdd-1162-4fc0-a945-0a7f95002d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.830676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-f850cbdd-1162-4fc0-a945-0a7f95002d92 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:24.831878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f850cbdd-1162-4fc0-a945-0a7f95002d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:24.871959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988931f8-fecf-47bb-a223-d8527fa15c98 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:24.916885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e554b-07f9-4395-9825-339fcf4d836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:24.917916 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e554b-07f9-4395-9825-339fcf4d836a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:24.918405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e554b-07f9-4395-9825-339fcf4d836a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:24.919675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e554b-07f9-4395-9825-339fcf4d836a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:24.920720 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e554b-07f9-4395-9825-339fcf4d836a None admin] RBAC: Adding query filter params (group.id=cf32feb4691047099d7e3ea98965d01f, role.id=82ee29c384454b9aa4db085c9d2e21e8, scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:24.921183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e554b-07f9-4395-9825-339fcf4d836a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:24.922799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e554b-07f9-4395-9825-339fcf4d836a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:24.930330 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 185/363] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Jul 25 08:32:24 2022] GET /identity/v3/role_assignments?group.id=cf32feb4691047099d7e3ea98965d01f&scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb&role.id=82ee29c384454b9aa4db085c9d2e21e8 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:24.931061 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 179/364] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:24.931519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e554b-07f9-4395-9825-339fcf4d836a None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:24.936619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850cbdd-1162-4fc0-a945-0a7f95002d92 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:25.016607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:25.048980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:25.080270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850cbdd-1162-4fc0-a945-0a7f95002d92 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:25.106233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e554b-07f9-4395-9825-339fcf4d836a None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:25.130330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00018fce-4676-4438-afc9-41aef87fa57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:25.131716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00018fce-4676-4438-afc9-41aef87fa57b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:25.132015 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00018fce-4676-4438-afc9-41aef87fa57b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:25.132296 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00018fce-4676-4438-afc9-41aef87fa57b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:25.145942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00018fce-4676-4438-afc9-41aef87fa57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-00018fce-4676-4438-afc9-41aef87fa57b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:25.154669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:25.155591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:25.156396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 180/365] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:25.157107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00018fce-4676-4438-afc9-41aef87fa57b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:25.157960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aacb67-b721-4627-b02d-8b9bd9605e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:25.158935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aacb67-b721-4627-b02d-8b9bd9605e5a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:25.159458 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aacb67-b721-4627-b02d-8b9bd9605e5a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:25.159934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aacb67-b721-4627-b02d-8b9bd9605e5a None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:25.160971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aacb67-b721-4627-b02d-8b9bd9605e5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:25.161485 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aacb67-b721-4627-b02d-8b9bd9605e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:25.163120 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aacb67-b721-4627-b02d-8b9bd9605e5a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:25.174260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 186/366] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:25.175402 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aacb67-b721-4627-b02d-8b9bd9605e5a None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:25.288844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00018fce-4676-4438-afc9-41aef87fa57b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:25.298280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aacb67-b721-4627-b02d-8b9bd9605e5a None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:25.311478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:25.324024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:25.332670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e6c8c-159c-45f0-b3e5-38c0d14823ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:25.334129 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e6c8c-159c-45f0-b3e5-38c0d14823ca None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:25.334619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e6c8c-159c-45f0-b3e5-38c0d14823ca None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:25.335118 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e6c8c-159c-45f0-b3e5-38c0d14823ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:25.336216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e6c8c-159c-45f0-b3e5-38c0d14823ca None admin] RBAC: Adding query filter params (group.id=cf32feb4691047099d7e3ea98965d01f, role.id=82ee29c384454b9aa4db085c9d2e21e8, scope.project.id=d68d17929cf4421f977087a339234c2a) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:25.336868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e6c8c-159c-45f0-b3e5-38c0d14823ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:25.338612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e6c8c-159c-45f0-b3e5-38c0d14823ca None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:25.343306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf4049-e8e3-4837-a67b-4c0f7c9f8c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:25.344116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf4049-e8e3-4837-a67b-4c0f7c9f8c17 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:25.344396 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf4049-e8e3-4837-a67b-4c0f7c9f8c17 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:25.344731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf4049-e8e3-4837-a67b-4c0f7c9f8c17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:25.346149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 181/367] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Jul 25 08:32:25 2022] GET /identity/v3/role_assignments?group.id=cf32feb4691047099d7e3ea98965d01f&scope.project.id=d68d17929cf4421f977087a339234c2a&role.id=82ee29c384454b9aa4db085c9d2e21e8 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:25.347166 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e6c8c-159c-45f0-b3e5-38c0d14823ca None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:25.370953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf4049-e8e3-4837-a67b-4c0f7c9f8c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-9adf4049-e8e3-4837-a67b-4c0f7c9f8c17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:25.381736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:25.382688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:25.383471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:25.383471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:25.383471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:25.383471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:25.383471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:25.383471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:25.383471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:25.383471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:25.383471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:25.383471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:25.383471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:25.383471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:25.385954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 187/368] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:32:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:25.386835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf4049-e8e3-4837-a67b-4c0f7c9f8c17 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:25.501887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e6c8c-159c-45f0-b3e5-38c0d14823ca None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:25.577320 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fefb3-3a5d-4c14-971d-6371d482d64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:25.578411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fefb3-3a5d-4c14-971d-6371d482d64e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:25.578924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fefb3-3a5d-4c14-971d-6371d482d64e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:25.579422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fefb3-3a5d-4c14-971d-6371d482d64e None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:25.581051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fefb3-3a5d-4c14-971d-6371d482d64e None admin] RBAC: Adding query filter params (name=reader) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:25.581385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fefb3-3a5d-4c14-971d-6371d482d64e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:25.582822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fefb3-3a5d-4c14-971d-6371d482d64e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:25.588629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf4049-e8e3-4837-a67b-4c0f7c9f8c17 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:25.594516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 182/369] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:25.594516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fefb3-3a5d-4c14-971d-6371d482d64e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:25.633017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82addb-740f-44d5-a3d8-7546ecb140ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:25.634235 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82addb-740f-44d5-a3d8-7546ecb140ab None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:25.634747 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82addb-740f-44d5-a3d8-7546ecb140ab None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:25.635224 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82addb-740f-44d5-a3d8-7546ecb140ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:25.671144 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:25.715434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82addb-740f-44d5-a3d8-7546ecb140ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:25.717480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82addb-740f-44d5-a3d8-7546ecb140ab None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:25.720072 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:25.753837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fefb3-3a5d-4c14-971d-6371d482d64e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:25.798741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5495b1-a892-4e15-b6ba-7991ec05ba46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:25.799929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5495b1-a892-4e15-b6ba-7991ec05ba46 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:25.799929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5495b1-a892-4e15-b6ba-7991ec05ba46 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:25.799929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5495b1-a892-4e15-b6ba-7991ec05ba46 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:25.800731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5495b1-a892-4e15-b6ba-7991ec05ba46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:25.801037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5495b1-a892-4e15-b6ba-7991ec05ba46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:25.802476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5495b1-a892-4e15-b6ba-7991ec05ba46 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:25.815710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 183/370] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:25.816566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5495b1-a892-4e15-b6ba-7991ec05ba46 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:25.836646 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 188/371] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:25.837852 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82addb-740f-44d5-a3d8-7546ecb140ab None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:25.930756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:25.941731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5495b1-a892-4e15-b6ba-7991ec05ba46 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:25.950938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82addb-740f-44d5-a3d8-7546ecb140ab None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:25.959712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:25.990837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212455e0-85a2-4a1b-b766-87f1d609dbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:25.991841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212455e0-85a2-4a1b-b766-87f1d609dbb3 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:25.992386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212455e0-85a2-4a1b-b766-87f1d609dbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:25.992890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212455e0-85a2-4a1b-b766-87f1d609dbb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:26.009311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0dc2a4-ba58-4dbe-bbdd-0de9d50fcf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:26.010221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dc2a4-ba58-4dbe-bbdd-0de9d50fcf58 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:26.010504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dc2a4-ba58-4dbe-bbdd-0de9d50fcf58 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:26.010777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dc2a4-ba58-4dbe-bbdd-0de9d50fcf58 None admin] PATH_INFO: `/v3/users/1c24431a118f4506aa9e64972bc31f00` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:26.012038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0dc2a4-ba58-4dbe-bbdd-0de9d50fcf58 None admin] RBAC: Authorizing `identity:get_user(user_id=1c24431a118f4506aa9e64972bc31f00)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:26.013567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0dc2a4-ba58-4dbe-bbdd-0de9d50fcf58 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:26.015428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 184/372] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:32:25 2022] GET /identity/v3/users/1c24431a118f4506aa9e64972bc31f00 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:26.016067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0dc2a4-ba58-4dbe-bbdd-0de9d50fcf58 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:26.099796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212455e0-85a2-4a1b-b766-87f1d609dbb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:26.102424 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212455e0-85a2-4a1b-b766-87f1d609dbb3 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:26.199532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0dc2a4-ba58-4dbe-bbdd-0de9d50fcf58 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:26.252752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:26.256272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83639dce-569a-4833-9f4d-32c59e8c09bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:26.257171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83639dce-569a-4833-9f4d-32c59e8c09bb None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:26.257450 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83639dce-569a-4833-9f4d-32c59e8c09bb None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:26.257728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83639dce-569a-4833-9f4d-32c59e8c09bb None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:26.259069 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83639dce-569a-4833-9f4d-32c59e8c09bb None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:26.260500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83639dce-569a-4833-9f4d-32c59e8c09bb None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:26.262218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 185/373] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:26 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:26.262977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83639dce-569a-4833-9f4d-32c59e8c09bb None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:26.296562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 189/374] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:26.297464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212455e0-85a2-4a1b-b766-87f1d609dbb3 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:26.375846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:26.464659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83639dce-569a-4833-9f4d-32c59e8c09bb None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:26.494082 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212455e0-85a2-4a1b-b766-87f1d609dbb3 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:26.528090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef4c7e-a296-40b2-b493-e39c40eb4878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:26.528903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef4c7e-a296-40b2-b493-e39c40eb4878 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:26.529186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef4c7e-a296-40b2-b493-e39c40eb4878 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:26.529714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef4c7e-a296-40b2-b493-e39c40eb4878 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:26.530562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef4c7e-a296-40b2-b493-e39c40eb4878 None admin] RBAC: Adding query filter params (role.id=fdcf2d1a66ab4deba04f767592b8c5df, user.id=1c24431a118f4506aa9e64972bc31f00, scope.system=all) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:26.530871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef4c7e-a296-40b2-b493-e39c40eb4878 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:26.532299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef4c7e-a296-40b2-b493-e39c40eb4878 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:26.546904 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 186/375] 10.222.0.52 () {56 vars in 1308 bytes} [Mon Jul 25 08:32:26 2022] GET /identity/v3/role_assignments?user.id=1c24431a118f4506aa9e64972bc31f00&scope.system=all&role.id=fdcf2d1a66ab4deba04f767592b8c5df => generated 510 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:26.547732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef4c7e-a296-40b2-b493-e39c40eb4878 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:26.562853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbaf77c-2657-4880-9381-db4fda5fe717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:26.563742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaf77c-2657-4880-9381-db4fda5fe717 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:26.564025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaf77c-2657-4880-9381-db4fda5fe717 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:26.564303 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaf77c-2657-4880-9381-db4fda5fe717 None admin] PATH_INFO: `/v3/groups/cf32feb4691047099d7e3ea98965d01f` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:26.565591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbaf77c-2657-4880-9381-db4fda5fe717 None admin] RBAC: Authorizing `identity:get_group(group_id=cf32feb4691047099d7e3ea98965d01f)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:26.567068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbaf77c-2657-4880-9381-db4fda5fe717 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:26.571818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 190/376] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:26 2022] GET /identity/v3/groups/cf32feb4691047099d7e3ea98965d01f => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:26.576695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbaf77c-2657-4880-9381-db4fda5fe717 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:26.649068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:26.751883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef4c7e-a296-40b2-b493-e39c40eb4878 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:26.756531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:26.758887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbaf77c-2657-4880-9381-db4fda5fe717 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:26.809116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2747e-9380-485e-a997-5224b902c3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:26.810353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2747e-9380-485e-a997-5224b902c3a9 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:26.810844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2747e-9380-485e-a997-5224b902c3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:26.811326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2747e-9380-485e-a997-5224b902c3a9 None admin] PATH_INFO: `/v3/projects/e86d2d432f134fa4aeaa99c3935e4c0c` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:26.815765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2747e-9380-485e-a997-5224b902c3a9 None admin] RBAC: Authorizing `identity:get_project(project_id=e86d2d432f134fa4aeaa99c3935e4c0c)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:26.818412 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2747e-9380-485e-a997-5224b902c3a9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:26.820917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 187/377] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:26 2022] GET /identity/v3/projects/e86d2d432f134fa4aeaa99c3935e4c0c => generated 344 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:26.828189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2747e-9380-485e-a997-5224b902c3a9 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:26.836598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a8898-01d7-4b3b-a963-5c231a9649a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:26.837298 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a8898-01d7-4b3b-a963-5c231a9649a1 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:26.837585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a8898-01d7-4b3b-a963-5c231a9649a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:26.837866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a8898-01d7-4b3b-a963-5c231a9649a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:26.838732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a8898-01d7-4b3b-a963-5c231a9649a1 None admin] RBAC: Adding query filter params (group.id=cf32feb4691047099d7e3ea98965d01f, role.id=3ce6c4ec0ed142828cb20882e342328e, scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:26.839049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a8898-01d7-4b3b-a963-5c231a9649a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:26.845668 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a8898-01d7-4b3b-a963-5c231a9649a1 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:26.859372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 191/378] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Jul 25 08:32:26 2022] GET /identity/v3/role_assignments?group.id=cf32feb4691047099d7e3ea98965d01f&scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb&role.id=3ce6c4ec0ed142828cb20882e342328e => generated 260 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:26.909154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676e9b9-8f67-44fe-982a-02ab42172737 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:26.909333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676e9b9-8f67-44fe-982a-02ab42172737 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:26.909615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676e9b9-8f67-44fe-982a-02ab42172737 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:26.912672 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 192/379] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:32:26.917239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996b6e5-9f99-4cd0-bfdd-43da809d9f9e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:26.917524 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996b6e5-9f99-4cd0-bfdd-43da809d9f9e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:26.917844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996b6e5-9f99-4cd0-bfdd-43da809d9f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:26.938123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-2996b6e5-9f99-4cd0-bfdd-43da809d9f9e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:26.939324 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2996b6e5-9f99-4cd0-bfdd-43da809d9f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:27.033491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2747e-9380-485e-a997-5224b902c3a9 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:27.075190 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 193/380] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:27.095564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a9136-8e83-46a0-9a89-b81febcbec2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:27.096884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a9136-8e83-46a0-9a89-b81febcbec2f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:27.097378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a9136-8e83-46a0-9a89-b81febcbec2f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:27.097856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a9136-8e83-46a0-9a89-b81febcbec2f None admin] PATH_INFO: `/v3/groups/5860b923e33c461f87ef80dfeb37b59d` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:27.106587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a9136-8e83-46a0-9a89-b81febcbec2f None admin] RBAC: Authorizing `identity:get_group(group_id=5860b923e33c461f87ef80dfeb37b59d)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:27.108439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a9136-8e83-46a0-9a89-b81febcbec2f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:27.108549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996b6e5-9f99-4cd0-bfdd-43da809d9f9e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:27.110993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 188/381] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:26 2022] GET /identity/v3/groups/5860b923e33c461f87ef80dfeb37b59d => generated 229 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:27.121759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a9136-8e83-46a0-9a89-b81febcbec2f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:27.169680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:27.172757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:27.245903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996b6e5-9f99-4cd0-bfdd-43da809d9f9e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:27.256267 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a9136-8e83-46a0-9a89-b81febcbec2f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:27.285363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0197e1-b979-43a2-b684-7757d478db02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:27.286658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0197e1-b979-43a2-b684-7757d478db02 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:27.287656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0197e1-b979-43a2-b684-7757d478db02 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:27.288136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0197e1-b979-43a2-b684-7757d478db02 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:27.303433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc9906-e12f-4d29-9905-d4eb3b716676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:27.303433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc9906-e12f-4d29-9905-d4eb3b716676 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:27.303433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc9906-e12f-4d29-9905-d4eb3b716676 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:27.303433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc9906-e12f-4d29-9905-d4eb3b716676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:27.303828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc9906-e12f-4d29-9905-d4eb3b716676 None admin] RBAC: Adding query filter params (group.id=5860b923e33c461f87ef80dfeb37b59d, role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=e86d2d432f134fa4aeaa99c3935e4c0c) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:27.304077 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc9906-e12f-4d29-9905-d4eb3b716676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:27.305561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc9906-e12f-4d29-9905-d4eb3b716676 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:27.308086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0197e1-b979-43a2-b684-7757d478db02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:27.313107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 189/382] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Jul 25 08:32:27 2022] GET /identity/v3/role_assignments?group.id=5860b923e33c461f87ef80dfeb37b59d&scope.project.id=e86d2d432f134fa4aeaa99c3935e4c0c&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-2a0197e1-b979-43a2-b684-7757d478db02 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:27.322878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:27.323848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:27.324702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:27.324702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:27.324702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:27.324702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:27.324702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:27.324702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:27.324702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:27.324702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:27.324702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:27.324702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:27.324702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:27.324702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:27.327719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 194/383] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:32:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:27.333170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc9906-e12f-4d29-9905-d4eb3b716676 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:27.393421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb4eb-bbf6-40bd-b3f2-7d378b51d24f None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:27.394043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb4eb-bbf6-40bd-b3f2-7d378b51d24f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:27.394525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb4eb-bbf6-40bd-b3f2-7d378b51d24f None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:27.396677 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 195/384] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:32:27.436762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643f059-5dee-4034-b6d7-455d168a80f7 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:27.437371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643f059-5dee-4034-b6d7-455d168a80f7 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:27.437852 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643f059-5dee-4034-b6d7-455d168a80f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:27.467927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d643f059-5dee-4034-b6d7-455d168a80f7 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:27.471161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d643f059-5dee-4034-b6d7-455d168a80f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:27.496255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:27.503500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc9906-e12f-4d29-9905-d4eb3b716676 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:27.564562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e4d7-b559-4859-bbed-7f9287fc40ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:27.564562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e4d7-b559-4859-bbed-7f9287fc40ec None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:27.564562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e4d7-b559-4859-bbed-7f9287fc40ec None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:27.564562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e4d7-b559-4859-bbed-7f9287fc40ec None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:27.566814 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640e4d7-b559-4859-bbed-7f9287fc40ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:27.566814 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640e4d7-b559-4859-bbed-7f9287fc40ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:27.568803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640e4d7-b559-4859-bbed-7f9287fc40ec None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:27.577798 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 190/385] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:27.582767 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e4d7-b559-4859-bbed-7f9287fc40ec None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:27.604501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 196/386] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:27.629271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643f059-5dee-4034-b6d7-455d168a80f7 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:27.703465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:27.707674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e4d7-b559-4859-bbed-7f9287fc40ec None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:27.754121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e059-d544-4bb9-ac39-1c63adfb2d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:27.755063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e059-d544-4bb9-ac39-1c63adfb2d81 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:27.755913 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e059-d544-4bb9-ac39-1c63adfb2d81 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:27.756195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e059-d544-4bb9-ac39-1c63adfb2d81 None admin] PATH_INFO: `/v3/groups/cf32feb4691047099d7e3ea98965d01f` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:27.757595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9e059-d544-4bb9-ac39-1c63adfb2d81 None admin] RBAC: Authorizing `identity:get_group(group_id=cf32feb4691047099d7e3ea98965d01f)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:27.759072 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9e059-d544-4bb9-ac39-1c63adfb2d81 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:27.760899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 191/387] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:27 2022] GET /identity/v3/groups/cf32feb4691047099d7e3ea98965d01f => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:27.770566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e059-d544-4bb9-ac39-1c63adfb2d81 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:27.827676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:27.846463 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643f059-5dee-4034-b6d7-455d168a80f7 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:27.889560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81b7d8-760a-41ad-872a-ddd983185529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:27.890537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81b7d8-760a-41ad-872a-ddd983185529 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:27.890927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81b7d8-760a-41ad-872a-ddd983185529 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:27.891312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81b7d8-760a-41ad-872a-ddd983185529 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:27.904675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81b7d8-760a-41ad-872a-ddd983185529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-9d81b7d8-760a-41ad-872a-ddd983185529 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:27.918239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:27.919157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:27.919939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:27.919939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:27.919939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:27.919939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:27.919939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:27.919939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:27.919939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:27.919939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:27.919939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:27.919939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:27.919939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:27.919939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:27.922135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 197/388] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:27.947213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81b7d8-760a-41ad-872a-ddd983185529 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:27.999406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e059-d544-4bb9-ac39-1c63adfb2d81 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:28.042330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658ab08-7507-4934-998e-c28c44d2ef26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:28.043211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658ab08-7507-4934-998e-c28c44d2ef26 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:28.043492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658ab08-7507-4934-998e-c28c44d2ef26 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:28.043765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658ab08-7507-4934-998e-c28c44d2ef26 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:28.045202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658ab08-7507-4934-998e-c28c44d2ef26 None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:28.046656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658ab08-7507-4934-998e-c28c44d2ef26 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:28.048705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 192/389] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:27 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:28.061848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658ab08-7507-4934-998e-c28c44d2ef26 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:28.128962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:28.146905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81b7d8-760a-41ad-872a-ddd983185529 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:28.218541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:28.221290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a30c4-cfa7-4d81-a3db-185371bbe15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:28.221979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a30c4-cfa7-4d81-a3db-185371bbe15f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:28.222409 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a30c4-cfa7-4d81-a3db-185371bbe15f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:28.222702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a30c4-cfa7-4d81-a3db-185371bbe15f None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:28.223487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a30c4-cfa7-4d81-a3db-185371bbe15f None admin] RBAC: Adding query filter params (name=member) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:28.223887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a30c4-cfa7-4d81-a3db-185371bbe15f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:28.225625 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a30c4-cfa7-4d81-a3db-185371bbe15f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:28.227197 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658ab08-7507-4934-998e-c28c44d2ef26 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:28.243060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 198/390] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:28.251487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a30c4-cfa7-4d81-a3db-185371bbe15f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:28.278812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee5413-d587-4bfe-a3b5-28aff50bf8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:28.281159 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee5413-d587-4bfe-a3b5-28aff50bf8e5 None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:28.281632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee5413-d587-4bfe-a3b5-28aff50bf8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:28.282317 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee5413-d587-4bfe-a3b5-28aff50bf8e5 None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a/groups/cf32feb4691047099d7e3ea98965d01f/roles/3ce6c4ec0ed142828cb20882e342328e` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:28.288203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee5413-d587-4bfe-a3b5-28aff50bf8e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d68d17929cf4421f977087a339234c2a, group_id=cf32feb4691047099d7e3ea98965d01f, role_id=3ce6c4ec0ed142828cb20882e342328e)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:28.290919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee5413-d587-4bfe-a3b5-28aff50bf8e5 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:28.322539 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 193/391] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Jul 25 08:32:28 2022] PUT /identity/v3/projects/d68d17929cf4421f977087a339234c2a/groups/cf32feb4691047099d7e3ea98965d01f/roles/3ce6c4ec0ed142828cb20882e342328e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:28.357816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bedc5-89fd-40a5-a777-4e0e6640a23a None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:28.358103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bedc5-89fd-40a5-a777-4e0e6640a23a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:28.358410 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bedc5-89fd-40a5-a777-4e0e6640a23a None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:28.363624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 194/392] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:32:28.367873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca37318-a644-43e9-9e6b-ccfbfbf2e121 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:28.367873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca37318-a644-43e9-9e6b-ccfbfbf2e121 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:28.367873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca37318-a644-43e9-9e6b-ccfbfbf2e121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:28.388962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-aca37318-a644-43e9-9e6b-ccfbfbf2e121 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:28.390236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-aca37318-a644-43e9-9e6b-ccfbfbf2e121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:28.478626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a30c4-cfa7-4d81-a3db-185371bbe15f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:28.488705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 195/393] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:28.517081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca37318-a644-43e9-9e6b-ccfbfbf2e121 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:28.566400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0fe0e-07c0-4831-8e77-47fbf0b76861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:28.567586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0fe0e-07c0-4831-8e77-47fbf0b76861 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:28.568077 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0fe0e-07c0-4831-8e77-47fbf0b76861 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:28.568589 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0fe0e-07c0-4831-8e77-47fbf0b76861 None admin] PATH_INFO: `/v3/groups/cf32feb4691047099d7e3ea98965d01f` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:28.573436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c0fe0e-07c0-4831-8e77-47fbf0b76861 None admin] RBAC: Authorizing `identity:get_group(group_id=cf32feb4691047099d7e3ea98965d01f)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:28.575155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c0fe0e-07c0-4831-8e77-47fbf0b76861 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:28.577473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 199/394] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:28 2022] GET /identity/v3/groups/cf32feb4691047099d7e3ea98965d01f => generated 226 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:28.584001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0fe0e-07c0-4831-8e77-47fbf0b76861 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:28.633236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:28.637685 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:28.650852 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca37318-a644-43e9-9e6b-ccfbfbf2e121 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:28.708054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a53bc-a7b5-4402-810a-7e609392461b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:28.710089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a53bc-a7b5-4402-810a-7e609392461b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:28.710600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a53bc-a7b5-4402-810a-7e609392461b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:28.711023 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a53bc-a7b5-4402-810a-7e609392461b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:28.726892 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a53bc-a7b5-4402-810a-7e609392461b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:28.736866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0fe0e-07c0-4831-8e77-47fbf0b76861 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-109a53bc-a7b5-4402-810a-7e609392461b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:28.737985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:28.739076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:28.740115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:28.740115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:28.740115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:28.740115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:28.740115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:28.740115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:28.740115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:28.740115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:28.740115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:28.740115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:28.740115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:28.740115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:28.745201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 196/395] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:28.750653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a53bc-a7b5-4402-810a-7e609392461b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:28.806649 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae1024-8bcc-4a42-b406-f20ce2943967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:28.807451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae1024-8bcc-4a42-b406-f20ce2943967 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:28.807737 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae1024-8bcc-4a42-b406-f20ce2943967 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:28.808018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae1024-8bcc-4a42-b406-f20ce2943967 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:28.809380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae1024-8bcc-4a42-b406-f20ce2943967 None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:28.813037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae1024-8bcc-4a42-b406-f20ce2943967 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:28.815161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 200/396] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:28 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:28.820853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae1024-8bcc-4a42-b406-f20ce2943967 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:28.905664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a53bc-a7b5-4402-810a-7e609392461b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:28.979907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:28.984054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7067341-93cf-4c01-845e-08be0819750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:28.985065 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7067341-93cf-4c01-845e-08be0819750e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:28.985348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7067341-93cf-4c01-845e-08be0819750e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:28.985627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7067341-93cf-4c01-845e-08be0819750e None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:28.986392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7067341-93cf-4c01-845e-08be0819750e None admin] RBAC: Adding query filter params (name=member) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:28.986705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7067341-93cf-4c01-845e-08be0819750e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:28.988163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7067341-93cf-4c01-845e-08be0819750e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:29.000771 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae1024-8bcc-4a42-b406-f20ce2943967 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:29.011716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 197/397] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:29.018824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7067341-93cf-4c01-845e-08be0819750e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:29.034779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:29.070054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f94519-df17-4ff2-abe9-e8090ba95042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:29.070886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f94519-df17-4ff2-abe9-e8090ba95042 None admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:29.071169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f94519-df17-4ff2-abe9-e8090ba95042 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:29.071451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f94519-df17-4ff2-abe9-e8090ba95042 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb/groups/cf32feb4691047099d7e3ea98965d01f/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:29.075808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f94519-df17-4ff2-abe9-e8090ba95042 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a11c903d8ba43eaa621b95be9cb82cb, group_id=cf32feb4691047099d7e3ea98965d01f, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:29.077302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f94519-df17-4ff2-abe9-e8090ba95042 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:29.106892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 201/398] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Jul 25 08:32:28 2022] PUT /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb/groups/cf32feb4691047099d7e3ea98965d01f/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:29.162249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7067341-93cf-4c01-845e-08be0819750e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:29.198946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf36aa-3150-4423-b307-d005456c7ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:29.199824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf36aa-3150-4423-b307-d005456c7ef0 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:29.200110 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf36aa-3150-4423-b307-d005456c7ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:29.200387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf36aa-3150-4423-b307-d005456c7ef0 None admin] PATH_INFO: `/v3/groups/cf32feb4691047099d7e3ea98965d01f` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:29.201616 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf36aa-3150-4423-b307-d005456c7ef0 None admin] RBAC: Authorizing `identity:get_group(group_id=cf32feb4691047099d7e3ea98965d01f)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:29.203069 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf36aa-3150-4423-b307-d005456c7ef0 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:29.205034 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 198/399] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:29 2022] GET /identity/v3/groups/cf32feb4691047099d7e3ea98965d01f => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:29.213745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f94519-df17-4ff2-abe9-e8090ba95042 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:29.300647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81b74a-185d-4878-a6ba-8d4632dcf213 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:29.301013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81b74a-185d-4878-a6ba-8d4632dcf213 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:29.301209 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81b74a-185d-4878-a6ba-8d4632dcf213 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:29.302881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 199/400] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:29.308486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63964ff9-e871-437c-844f-5b23544d6cf4 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:29.309059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63964ff9-e871-437c-844f-5b23544d6cf4 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:29.309541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63964ff9-e871-437c-844f-5b23544d6cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:29.329676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-63964ff9-e871-437c-844f-5b23544d6cf4 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:29.331203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-63964ff9-e871-437c-844f-5b23544d6cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:29.398024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f94519-df17-4ff2-abe9-e8090ba95042 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:29.429567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 200/401] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:29.444050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd57f8e-fab9-4a56-8d79-b23664fa4ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:29.444954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd57f8e-fab9-4a56-8d79-b23664fa4ebe None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:29.445237 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd57f8e-fab9-4a56-8d79-b23664fa4ebe None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:29.445514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd57f8e-fab9-4a56-8d79-b23664fa4ebe None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:29.446714 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd57f8e-fab9-4a56-8d79-b23664fa4ebe None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:29.448154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd57f8e-fab9-4a56-8d79-b23664fa4ebe None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:29.450199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 202/402] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:29 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:29.457829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63964ff9-e871-437c-844f-5b23544d6cf4 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:29.462446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd57f8e-fab9-4a56-8d79-b23664fa4ebe None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:29.467714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:29.483494 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:29.588810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63964ff9-e871-437c-844f-5b23544d6cf4 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:29.610554 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd57f8e-fab9-4a56-8d79-b23664fa4ebe None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:29.621243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cead9-b886-449c-9e23-e22691775d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:29.622137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cead9-b886-449c-9e23-e22691775d0d None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:29.622419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cead9-b886-449c-9e23-e22691775d0d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:29.622697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cead9-b886-449c-9e23-e22691775d0d None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a/groups/cf32feb4691047099d7e3ea98965d01f/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:29.624796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cead9-b886-449c-9e23-e22691775d0d None admin] RBAC: Authorizing `identity:create_grant(project_id=d68d17929cf4421f977087a339234c2a, group_id=cf32feb4691047099d7e3ea98965d01f, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:29.626262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cead9-b886-449c-9e23-e22691775d0d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:29.649153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 201/403] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Jul 25 08:32:29 2022] PUT /identity/v3/projects/d68d17929cf4421f977087a339234c2a/groups/cf32feb4691047099d7e3ea98965d01f/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:29.706120 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f0406-f149-4a61-9915-720d0541d3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:29.706978 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f0406-f149-4a61-9915-720d0541d3b7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:29.707271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f0406-f149-4a61-9915-720d0541d3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:29.707551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f0406-f149-4a61-9915-720d0541d3b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:29.733397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f0406-f149-4a61-9915-720d0541d3b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-3b1f0406-f149-4a61-9915-720d0541d3b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:29.742582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:29.743529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:29.744956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 203/404] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:32:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:29.752454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cead9-b886-449c-9e23-e22691775d0d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:29.861471 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:29.902989 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cead9-b886-449c-9e23-e22691775d0d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:29.939745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac0917-0283-4ef8-a2d8-e8e3960d6e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:29.940374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac0917-0283-4ef8-a2d8-e8e3960d6e78 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:29.940689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac0917-0283-4ef8-a2d8-e8e3960d6e78 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:29.941014 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac0917-0283-4ef8-a2d8-e8e3960d6e78 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:29.942095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac0917-0283-4ef8-a2d8-e8e3960d6e78 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:29.942272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac0917-0283-4ef8-a2d8-e8e3960d6e78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:29.943653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac0917-0283-4ef8-a2d8-e8e3960d6e78 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:29.943877 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d6e7c-46c2-48da-b364-ee94510e3a70 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:29.944033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d6e7c-46c2-48da-b364-ee94510e3a70 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:29.944216 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d6e7c-46c2-48da-b364-ee94510e3a70 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:29.946020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 204/405] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:29.951026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce488a-7f3d-44e6-8478-30cface2eb02 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:29.951318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce488a-7f3d-44e6-8478-30cface2eb02 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:29.951605 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce488a-7f3d-44e6-8478-30cface2eb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:29.952940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 202/406] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:29.958151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac0917-0283-4ef8-a2d8-e8e3960d6e78 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:29.975906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-f7ce488a-7f3d-44e6-8478-30cface2eb02 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:29.977276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f7ce488a-7f3d-44e6-8478-30cface2eb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:30.056107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 205/407] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:30.080522 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac0917-0283-4ef8-a2d8-e8e3960d6e78 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:30.080739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce488a-7f3d-44e6-8478-30cface2eb02 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:30.092011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:30.115735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:30.129012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901ffe3-107d-43c7-9a09-efa8cccf6a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:30.130853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901ffe3-107d-43c7-9a09-efa8cccf6a99 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:30.131342 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901ffe3-107d-43c7-9a09-efa8cccf6a99 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:30.132322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901ffe3-107d-43c7-9a09-efa8cccf6a99 None admin] PATH_INFO: `/v3/groups/cf32feb4691047099d7e3ea98965d01f` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:30.134264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901ffe3-107d-43c7-9a09-efa8cccf6a99 None admin] RBAC: Authorizing `identity:get_group(group_id=cf32feb4691047099d7e3ea98965d01f)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:30.136638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901ffe3-107d-43c7-9a09-efa8cccf6a99 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:30.139540 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 203/408] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:29 2022] GET /identity/v3/groups/cf32feb4691047099d7e3ea98965d01f => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:30.145823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901ffe3-107d-43c7-9a09-efa8cccf6a99 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:30.235788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce488a-7f3d-44e6-8478-30cface2eb02 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:30.303970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cc824-9175-4d8b-a0db-246aee206c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:30.304807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cc824-9175-4d8b-a0db-246aee206c1c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:30.305094 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cc824-9175-4d8b-a0db-246aee206c1c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:30.305379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cc824-9175-4d8b-a0db-246aee206c1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:30.322318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cc824-9175-4d8b-a0db-246aee206c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:30.332858 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901ffe3-107d-43c7-9a09-efa8cccf6a99 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-f87cc824-9175-4d8b-a0db-246aee206c1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:30.335823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:30.336694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:30.337514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:30.337514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:30.337514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:30.337514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:30.337514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:30.337514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:30.337514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:30.337514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:30.337514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:30.337514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:30.337514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:30.337514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:30.343373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 206/409] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:32:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:30.349601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cc824-9175-4d8b-a0db-246aee206c1c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:30.386783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87122fee-4cd9-48a4-a03d-7fa8744e696f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:30.387664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87122fee-4cd9-48a4-a03d-7fa8744e696f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:30.387952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87122fee-4cd9-48a4-a03d-7fa8744e696f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:30.388227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87122fee-4cd9-48a4-a03d-7fa8744e696f None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:30.390213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87122fee-4cd9-48a4-a03d-7fa8744e696f None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:30.391719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87122fee-4cd9-48a4-a03d-7fa8744e696f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:30.396599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 204/410] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:30 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:30.401689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87122fee-4cd9-48a4-a03d-7fa8744e696f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:30.480090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:30.517992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:30.527600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cc824-9175-4d8b-a0db-246aee206c1c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:30.570618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581bad9-1205-4cc4-9069-3ae46ed4e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:30.571356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581bad9-1205-4cc4-9069-3ae46ed4e23a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:30.571650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581bad9-1205-4cc4-9069-3ae46ed4e23a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:30.571931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581bad9-1205-4cc4-9069-3ae46ed4e23a None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:30.572780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581bad9-1205-4cc4-9069-3ae46ed4e23a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:30.573093 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581bad9-1205-4cc4-9069-3ae46ed4e23a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:30.580596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581bad9-1205-4cc4-9069-3ae46ed4e23a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:30.585208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 207/411] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:30.591403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581bad9-1205-4cc4-9069-3ae46ed4e23a None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:30.618523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87122fee-4cd9-48a4-a03d-7fa8744e696f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:30.702721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd9eac1-1dfc-486a-918f-2ee2c0760982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:30.703868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9eac1-1dfc-486a-918f-2ee2c0760982 None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:30.704304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9eac1-1dfc-486a-918f-2ee2c0760982 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:30.704762 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9eac1-1dfc-486a-918f-2ee2c0760982 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb/groups/cf32feb4691047099d7e3ea98965d01f/roles/3ce6c4ec0ed142828cb20882e342328e` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:30.712032 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581bad9-1205-4cc4-9069-3ae46ed4e23a None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:30.713286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd9eac1-1dfc-486a-918f-2ee2c0760982 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a11c903d8ba43eaa621b95be9cb82cb, group_id=cf32feb4691047099d7e3ea98965d01f, role_id=3ce6c4ec0ed142828cb20882e342328e)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:30.714917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd9eac1-1dfc-486a-918f-2ee2c0760982 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:30.745501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:30.748748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 205/412] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Jul 25 08:32:30 2022] PUT /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb/groups/cf32feb4691047099d7e3ea98965d01f/roles/3ce6c4ec0ed142828cb20882e342328e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:32:30.759796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357bec7-aa90-46af-b84c-22b5bdd43c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:30.760848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357bec7-aa90-46af-b84c-22b5bdd43c6a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:30.761141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357bec7-aa90-46af-b84c-22b5bdd43c6a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:30.761477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357bec7-aa90-46af-b84c-22b5bdd43c6a None admin] PATH_INFO: `/v3/groups/5860b923e33c461f87ef80dfeb37b59d` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:30.762910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357bec7-aa90-46af-b84c-22b5bdd43c6a None admin] RBAC: Authorizing `identity:get_group(group_id=5860b923e33c461f87ef80dfeb37b59d)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:30.764423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357bec7-aa90-46af-b84c-22b5bdd43c6a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:30.766290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 208/413] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:30 2022] GET /identity/v3/groups/5860b923e33c461f87ef80dfeb37b59d => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:30.772164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd9eac1-1dfc-486a-918f-2ee2c0760982 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:30.917900 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9803f0-65f2-46d4-8fd6-6f70debfab81 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:30.918307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9803f0-65f2-46d4-8fd6-6f70debfab81 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:30.919162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9803f0-65f2-46d4-8fd6-6f70debfab81 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:30.920771 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 209/414] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:30.927238 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceae27c-ce3b-471f-9fae-15727db91e46 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:30.927537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceae27c-ce3b-471f-9fae-15727db91e46 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:30.928090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceae27c-ce3b-471f-9fae-15727db91e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:30.937461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd9eac1-1dfc-486a-918f-2ee2c0760982 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:30.950200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-2ceae27c-ce3b-471f-9fae-15727db91e46 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:30.951851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2ceae27c-ce3b-471f-9fae-15727db91e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:30.973287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:30.987556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4ce1a-7a85-4940-abd8-5188535b93ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:30.988472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4ce1a-7a85-4940-abd8-5188535b93ad None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:30.988819 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4ce1a-7a85-4940-abd8-5188535b93ad None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:30.989298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4ce1a-7a85-4940-abd8-5188535b93ad None admin] PATH_INFO: `/v3/projects/e86d2d432f134fa4aeaa99c3935e4c0c` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:30.990876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4ce1a-7a85-4940-abd8-5188535b93ad None admin] RBAC: Authorizing `identity:get_project(project_id=e86d2d432f134fa4aeaa99c3935e4c0c)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:30.992329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4ce1a-7a85-4940-abd8-5188535b93ad None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:30.994449 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 206/415] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:30 2022] GET /identity/v3/projects/e86d2d432f134fa4aeaa99c3935e4c0c => generated 344 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:31.000255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4ce1a-7a85-4940-abd8-5188535b93ad None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:31.041580 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 210/416] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:31.065716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceae27c-ce3b-471f-9fae-15727db91e46 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:31.108874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4ce1a-7a85-4940-abd8-5188535b93ad None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:31.143456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaa3b6-f5e9-450f-8936-afec6e81c0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:31.144356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaa3b6-f5e9-450f-8936-afec6e81c0ce None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:31.144668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaa3b6-f5e9-450f-8936-afec6e81c0ce None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:31.144957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaa3b6-f5e9-450f-8936-afec6e81c0ce None admin] PATH_INFO: `/v3/projects/e86d2d432f134fa4aeaa99c3935e4c0c/groups/5860b923e33c461f87ef80dfeb37b59d/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:31.147523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aaa3b6-f5e9-450f-8936-afec6e81c0ce None admin] RBAC: Authorizing `identity:create_grant(project_id=e86d2d432f134fa4aeaa99c3935e4c0c, group_id=5860b923e33c461f87ef80dfeb37b59d, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:31.149022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aaa3b6-f5e9-450f-8936-afec6e81c0ce None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:31.169646 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 207/417] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Jul 25 08:32:30 2022] PUT /identity/v3/projects/e86d2d432f134fa4aeaa99c3935e4c0c/groups/5860b923e33c461f87ef80dfeb37b59d/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:32:31.183293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceae27c-ce3b-471f-9fae-15727db91e46 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:31.218415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:31.229037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679263e-7d5d-430c-b731-f09b4d87732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:31.230155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679263e-7d5d-430c-b731-f09b4d87732d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:31.230591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679263e-7d5d-430c-b731-f09b4d87732d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:31.231031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679263e-7d5d-430c-b731-f09b4d87732d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:31.244202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679263e-7d5d-430c-b731-f09b4d87732d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-8679263e-7d5d-430c-b731-f09b4d87732d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:31.253542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:31.254700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:31.256203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:31.256203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:31.256203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:31.256203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:31.256203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:31.256203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:31.256203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:31.256203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:31.256203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:31.256203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:31.256203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:31.256203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:31.258335 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 211/418] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:32:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:31.264518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaa3b6-f5e9-450f-8936-afec6e81c0ce None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:31.386069 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:31.440728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaa3b6-f5e9-450f-8936-afec6e81c0ce None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:31.478582 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92ccd6-5101-4cd4-ba18-ed58e7811c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:31.479323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92ccd6-5101-4cd4-ba18-ed58e7811c02 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:31.479547 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92ccd6-5101-4cd4-ba18-ed58e7811c02 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:31.479826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92ccd6-5101-4cd4-ba18-ed58e7811c02 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:31.480624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92ccd6-5101-4cd4-ba18-ed58e7811c02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:31.480916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92ccd6-5101-4cd4-ba18-ed58e7811c02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:31.482300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92ccd6-5101-4cd4-ba18-ed58e7811c02 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:31.492644 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 208/419] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:31.498174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679263e-7d5d-430c-b731-f09b4d87732d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:31.610276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f08c3b-811c-49bc-ab24-250add311056 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:31.610577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f08c3b-811c-49bc-ab24-250add311056 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:31.610867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f08c3b-811c-49bc-ab24-250add311056 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:31.612512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 209/420] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:31.617237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8163c2-e1d5-4452-817b-8effb7880029 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:31.617526 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8163c2-e1d5-4452-817b-8effb7880029 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:31.617809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8163c2-e1d5-4452-817b-8effb7880029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:31.637399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-4e8163c2-e1d5-4452-817b-8effb7880029 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:31.638584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4e8163c2-e1d5-4452-817b-8effb7880029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:31.652737 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679263e-7d5d-430c-b731-f09b4d87732d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:31.722784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b257b6-b6bb-49c3-8b39-53d2edda3886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:31.725901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b257b6-b6bb-49c3-8b39-53d2edda3886 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:31.726402 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b257b6-b6bb-49c3-8b39-53d2edda3886 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:31.726904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b257b6-b6bb-49c3-8b39-53d2edda3886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:31.767528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 210/421] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:31.792450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:31.803418 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8163c2-e1d5-4452-817b-8effb7880029 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:31.822171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b257b6-b6bb-49c3-8b39-53d2edda3886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:31.823630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b257b6-b6bb-49c3-8b39-53d2edda3886 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:31.870025 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:31.950837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8163c2-e1d5-4452-817b-8effb7880029 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:32.001739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 212/422] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:32.007188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b257b6-b6bb-49c3-8b39-53d2edda3886 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:32.020648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afe062-3dd2-4d1d-85f0-c3e5782898c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:32.021531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afe062-3dd2-4d1d-85f0-c3e5782898c4 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:32.021814 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afe062-3dd2-4d1d-85f0-c3e5782898c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:32.022096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afe062-3dd2-4d1d-85f0-c3e5782898c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:32.042727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afe062-3dd2-4d1d-85f0-c3e5782898c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-a1afe062-3dd2-4d1d-85f0-c3e5782898c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:32.053650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:32.054783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:32.055769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:32.055769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:32.055769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:32.055769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:32.055769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:32.055769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:32.055769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:32.055769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:32.055769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:32.055769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:32.055769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:32.055769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:32.056511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 211/423] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:32.065955 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afe062-3dd2-4d1d-85f0-c3e5782898c4 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:32.165029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:32.178118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b257b6-b6bb-49c3-8b39-53d2edda3886 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:32.229702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afe062-3dd2-4d1d-85f0-c3e5782898c4 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:32.232457 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:32.252700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150f841-05bc-410d-820e-8d3bfdd3b4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:32.253579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150f841-05bc-410d-820e-8d3bfdd3b4ec None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:32.253862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150f841-05bc-410d-820e-8d3bfdd3b4ec None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:32.254135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150f841-05bc-410d-820e-8d3bfdd3b4ec None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:32.255884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150f841-05bc-410d-820e-8d3bfdd3b4ec None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:32.257337 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150f841-05bc-410d-820e-8d3bfdd3b4ec None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:32.259407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 213/424] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:32 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:32.264720 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150f841-05bc-410d-820e-8d3bfdd3b4ec None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:32.287303 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cef92-030f-404a-8dba-a39dc15940a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:32.293098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cef92-030f-404a-8dba-a39dc15940a9 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:32.293421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cef92-030f-404a-8dba-a39dc15940a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:32.293710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cef92-030f-404a-8dba-a39dc15940a9 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:32.294502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55cef92-030f-404a-8dba-a39dc15940a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:32.294819 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55cef92-030f-404a-8dba-a39dc15940a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:32.296215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55cef92-030f-404a-8dba-a39dc15940a9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:32.311292 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 212/425] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:32.321259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cef92-030f-404a-8dba-a39dc15940a9 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:32.447119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150f841-05bc-410d-820e-8d3bfdd3b4ec None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:32.487714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cef92-030f-404a-8dba-a39dc15940a9 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:32.491122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee812e2-a6b3-48ce-9dcf-40eb6814c7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:32.492000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee812e2-a6b3-48ce-9dcf-40eb6814c7fc None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:32.492287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee812e2-a6b3-48ce-9dcf-40eb6814c7fc None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:32.492586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee812e2-a6b3-48ce-9dcf-40eb6814c7fc None admin] PATH_INFO: `/v3/groups/cf32feb4691047099d7e3ea98965d01f` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:32.493789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee812e2-a6b3-48ce-9dcf-40eb6814c7fc None admin] RBAC: Authorizing `identity:get_group(group_id=cf32feb4691047099d7e3ea98965d01f)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:32.495646 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee812e2-a6b3-48ce-9dcf-40eb6814c7fc None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:32.497692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 214/426] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:32 2022] GET /identity/v3/groups/cf32feb4691047099d7e3ea98965d01f => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:32.498883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945adb2d-87b9-4c33-8730-56e338a98344 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:32.499171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945adb2d-87b9-4c33-8730-56e338a98344 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:32.499452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945adb2d-87b9-4c33-8730-56e338a98344 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:32.500863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 215/427] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:32.506320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945adb2d-87b9-4c33-8730-56e338a98344 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:32.534898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9b76f-5fe4-4590-9b5a-c08d670e6326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:32.535604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9b76f-5fe4-4590-9b5a-c08d670e6326 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:32.536185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9b76f-5fe4-4590-9b5a-c08d670e6326 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:32.537133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9b76f-5fe4-4590-9b5a-c08d670e6326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:32.537758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:32.579988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:32.631095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9b76f-5fe4-4590-9b5a-c08d670e6326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:32.632863 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9b76f-5fe4-4590-9b5a-c08d670e6326 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:32.634314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945adb2d-87b9-4c33-8730-56e338a98344 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:32.676422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948ae7a-b800-450c-83f4-ff3580a78858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:32.677277 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948ae7a-b800-450c-83f4-ff3580a78858 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:32.677626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948ae7a-b800-450c-83f4-ff3580a78858 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:32.677966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948ae7a-b800-450c-83f4-ff3580a78858 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:32.679771 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948ae7a-b800-450c-83f4-ff3580a78858 None admin] RBAC: Adding query filter params (group.id=cf32feb4691047099d7e3ea98965d01f, role.id=3ce6c4ec0ed142828cb20882e342328e, scope.project.id=d68d17929cf4421f977087a339234c2a) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:32.680166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948ae7a-b800-450c-83f4-ff3580a78858 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:32.681684 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948ae7a-b800-450c-83f4-ff3580a78858 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:32.692769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 216/428] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Jul 25 08:32:32 2022] GET /identity/v3/role_assignments?group.id=cf32feb4691047099d7e3ea98965d01f&scope.project.id=d68d17929cf4421f977087a339234c2a&role.id=3ce6c4ec0ed142828cb20882e342328e => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:32.694194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf263ed-22a3-4e3f-bd7a-4c0e160a9088 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:32.694487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf263ed-22a3-4e3f-bd7a-4c0e160a9088 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:32.694766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf263ed-22a3-4e3f-bd7a-4c0e160a9088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:32.733667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-2cf263ed-22a3-4e3f-bd7a-4c0e160a9088 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:32.734888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2cf263ed-22a3-4e3f-bd7a-4c0e160a9088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:32.818754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 213/429] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:32.824203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9b76f-5fe4-4590-9b5a-c08d670e6326 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:32.837842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 217/430] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:32.866590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf263ed-22a3-4e3f-bd7a-4c0e160a9088 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:32.889345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:32.947930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:32.993709 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf263ed-22a3-4e3f-bd7a-4c0e160a9088 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:32.996020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9b76f-5fe4-4590-9b5a-c08d670e6326 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:33.043354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb970868-c73e-4352-a102-dc87f6624b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:33.044175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb970868-c73e-4352-a102-dc87f6624b40 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.044457 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb970868-c73e-4352-a102-dc87f6624b40 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.044814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb970868-c73e-4352-a102-dc87f6624b40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.047988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88501d-77b6-4df9-8d4d-7612c82202a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:33.048949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88501d-77b6-4df9-8d4d-7612c82202a9 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.049231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88501d-77b6-4df9-8d4d-7612c82202a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.049502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88501d-77b6-4df9-8d4d-7612c82202a9 None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.050729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88501d-77b6-4df9-8d4d-7612c82202a9 None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:33.052208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88501d-77b6-4df9-8d4d-7612c82202a9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:33.054269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 214/431] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:32 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:33.062915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb970868-c73e-4352-a102-dc87f6624b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-cb970868-c73e-4352-a102-dc87f6624b40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:33.071374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:33.072268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:33.073090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 218/432] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:33.074558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88501d-77b6-4df9-8d4d-7612c82202a9 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:33.078350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb970868-c73e-4352-a102-dc87f6624b40 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:33.171033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:33.184323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88501d-77b6-4df9-8d4d-7612c82202a9 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:33.196311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb970868-c73e-4352-a102-dc87f6624b40 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:33.220044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b049e-545f-4789-9aa2-5689a41bf193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:33.220964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b049e-545f-4789-9aa2-5689a41bf193 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.221247 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b049e-545f-4789-9aa2-5689a41bf193 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.221530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b049e-545f-4789-9aa2-5689a41bf193 None admin] PATH_INFO: `/v3/groups/cf32feb4691047099d7e3ea98965d01f` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.222731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b049e-545f-4789-9aa2-5689a41bf193 None admin] RBAC: Authorizing `identity:get_group(group_id=cf32feb4691047099d7e3ea98965d01f)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:33.224475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b049e-545f-4789-9aa2-5689a41bf193 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:33.226131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 215/433] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:33 2022] GET /identity/v3/groups/cf32feb4691047099d7e3ea98965d01f => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:33.226131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c35c-3f06-43f5-8d20-d2523983fdda None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.226131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c35c-3f06-43f5-8d20-d2523983fdda None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.226131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c35c-3f06-43f5-8d20-d2523983fdda None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.226131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 216/434] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:33.232901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8c35c-3f06-43f5-8d20-d2523983fdda None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:33.235653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:33.240989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2525d68-6a33-4c6a-90b9-d491421949fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:33.241676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2525d68-6a33-4c6a-90b9-d491421949fb None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.241966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2525d68-6a33-4c6a-90b9-d491421949fb None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.242247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2525d68-6a33-4c6a-90b9-d491421949fb None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.243445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2525d68-6a33-4c6a-90b9-d491421949fb None admin] RBAC: Adding query filter params (name=member) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:33.243758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2525d68-6a33-4c6a-90b9-d491421949fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:33.245541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2525d68-6a33-4c6a-90b9-d491421949fb None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:33.254867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 219/435] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:32:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:33.256527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e9559-7570-4abe-bdcb-9309b8c8086d None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.256850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e9559-7570-4abe-bdcb-9309b8c8086d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.257132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e9559-7570-4abe-bdcb-9309b8c8086d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.275526 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-f17e9559-7570-4abe-bdcb-9309b8c8086d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:33.276769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f17e9559-7570-4abe-bdcb-9309b8c8086d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:33.349254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8c35c-3f06-43f5-8d20-d2523983fdda None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:33.356680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 220/436] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:33.356680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e9559-7570-4abe-bdcb-9309b8c8086d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:33.390245 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f0bd3-ca9f-4576-ad71-e6850b03a14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:33.391003 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f0bd3-ca9f-4576-ad71-e6850b03a14b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.391297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f0bd3-ca9f-4576-ad71-e6850b03a14b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.391578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f0bd3-ca9f-4576-ad71-e6850b03a14b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.392418 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f0bd3-ca9f-4576-ad71-e6850b03a14b None admin] RBAC: Adding query filter params (group.id=cf32feb4691047099d7e3ea98965d01f, role.id=82ee29c384454b9aa4db085c9d2e21e8, scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:33.392787 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f0bd3-ca9f-4576-ad71-e6850b03a14b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:33.394222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f0bd3-ca9f-4576-ad71-e6850b03a14b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:33.400249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 217/437] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Jul 25 08:32:33 2022] GET /identity/v3/role_assignments?group.id=cf32feb4691047099d7e3ea98965d01f&scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb&role.id=82ee29c384454b9aa4db085c9d2e21e8 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:33.401081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f0bd3-ca9f-4576-ad71-e6850b03a14b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:33.457019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:33.477322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e9559-7570-4abe-bdcb-9309b8c8086d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:33.490861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:33.520952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186513c3-86fd-481b-b972-b6034f9e290c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:33.521797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186513c3-86fd-481b-b972-b6034f9e290c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.522093 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186513c3-86fd-481b-b972-b6034f9e290c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.522375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186513c3-86fd-481b-b972-b6034f9e290c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.535288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f0bd3-ca9f-4576-ad71-e6850b03a14b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:33.573880 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e15e7-749d-43c4-b918-a2e17b0cc14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:33.575019 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e15e7-749d-43c4-b918-a2e17b0cc14d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.575483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e15e7-749d-43c4-b918-a2e17b0cc14d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.575987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e15e7-749d-43c4-b918-a2e17b0cc14d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.588855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e15e7-749d-43c4-b918-a2e17b0cc14d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-ca4e15e7-749d-43c4-b918-a2e17b0cc14d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:33.597512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:33.598438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:33.599775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 218/438] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:32:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:33.601329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fce932-75df-499a-bcbf-1e82b6db1e9c None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.601329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fce932-75df-499a-bcbf-1e82b6db1e9c None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.601329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fce932-75df-499a-bcbf-1e82b6db1e9c None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.603132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 219/439] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:33.605646 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186513c3-86fd-481b-b972-b6034f9e290c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:33.607058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fce932-75df-499a-bcbf-1e82b6db1e9c None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:33.607184 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186513c3-86fd-481b-b972-b6034f9e290c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:33.685562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:33.715761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fce932-75df-499a-bcbf-1e82b6db1e9c None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:33.719178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 221/440] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:33.720435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70cf6d-d124-42b4-8cb6-b628e5228ddd None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.720758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70cf6d-d124-42b4-8cb6-b628e5228ddd None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.721036 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70cf6d-d124-42b4-8cb6-b628e5228ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.739380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7f70cf6d-d124-42b4-8cb6-b628e5228ddd None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:33.740590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7f70cf6d-d124-42b4-8cb6-b628e5228ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:33.741124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:33.753084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dccf66-af22-40ad-aad2-6e3aa1c963b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:33.754134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dccf66-af22-40ad-aad2-6e3aa1c963b3 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.754512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dccf66-af22-40ad-aad2-6e3aa1c963b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.755461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dccf66-af22-40ad-aad2-6e3aa1c963b3 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.756794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dccf66-af22-40ad-aad2-6e3aa1c963b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:33.757300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dccf66-af22-40ad-aad2-6e3aa1c963b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:33.759656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dccf66-af22-40ad-aad2-6e3aa1c963b3 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:33.769621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 220/441] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:33.770384 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dccf66-af22-40ad-aad2-6e3aa1c963b3 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:33.815328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 222/442] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:33.816400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70cf6d-d124-42b4-8cb6-b628e5228ddd None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:33.871034 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dccf66-af22-40ad-aad2-6e3aa1c963b3 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:33.905491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310aacf-6f3e-4880-a76b-522b75fb150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:33.906379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310aacf-6f3e-4880-a76b-522b75fb150f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.906659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310aacf-6f3e-4880-a76b-522b75fb150f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.906934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310aacf-6f3e-4880-a76b-522b75fb150f None admin] PATH_INFO: `/v3/projects/d68d17929cf4421f977087a339234c2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.908149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310aacf-6f3e-4880-a76b-522b75fb150f None admin] RBAC: Authorizing `identity:get_project(project_id=d68d17929cf4421f977087a339234c2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:33.909617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310aacf-6f3e-4880-a76b-522b75fb150f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:33.915638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 221/443] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:33 2022] GET /identity/v3/projects/d68d17929cf4421f977087a339234c2a => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:33.915638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310aacf-6f3e-4880-a76b-522b75fb150f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:33.917542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:33.920494 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70cf6d-d124-42b4-8cb6-b628e5228ddd None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:33.956661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1839e5-25fc-48f7-a5c1-88562e552b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:33.957309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1839e5-25fc-48f7-a5c1-88562e552b96 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:33.957596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1839e5-25fc-48f7-a5c1-88562e552b96 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:33.957872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1839e5-25fc-48f7-a5c1-88562e552b96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:33.968581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:34.012779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310aacf-6f3e-4880-a76b-522b75fb150f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:34.025663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1839e5-25fc-48f7-a5c1-88562e552b96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:34.027127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1839e5-25fc-48f7-a5c1-88562e552b96 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:34.046571 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c9e8d-bd8c-4baa-9345-b7ae880bede8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:34.047354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c9e8d-bd8c-4baa-9345-b7ae880bede8 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:34.047631 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c9e8d-bd8c-4baa-9345-b7ae880bede8 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:34.047898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c9e8d-bd8c-4baa-9345-b7ae880bede8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:34.058154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c9e8d-bd8c-4baa-9345-b7ae880bede8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-d57c9e8d-bd8c-4baa-9345-b7ae880bede8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:34.065728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:34.066808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:34.068161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 222/444] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:32:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:34.068856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c9e8d-bd8c-4baa-9345-b7ae880bede8 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:34.123656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:34.130054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 223/445] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:34.130054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1839e5-25fc-48f7-a5c1-88562e552b96 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:34.165968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c9e8d-bd8c-4baa-9345-b7ae880bede8 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:34.182407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:34.200650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4b5e6-9452-47ac-b7a9-b4d89ef0361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:34.201504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4b5e6-9452-47ac-b7a9-b4d89ef0361c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:34.201794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4b5e6-9452-47ac-b7a9-b4d89ef0361c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:34.202068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4b5e6-9452-47ac-b7a9-b4d89ef0361c None admin] PATH_INFO: `/v3/groups/cf32feb4691047099d7e3ea98965d01f` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:34.203248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4b5e6-9452-47ac-b7a9-b4d89ef0361c None admin] RBAC: Authorizing `identity:get_group(group_id=cf32feb4691047099d7e3ea98965d01f)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:34.204736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4b5e6-9452-47ac-b7a9-b4d89ef0361c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:34.206376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 223/446] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:34 2022] GET /identity/v3/groups/cf32feb4691047099d7e3ea98965d01f => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:34.207131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4b5e6-9452-47ac-b7a9-b4d89ef0361c None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:34.232512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1839e5-25fc-48f7-a5c1-88562e552b96 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:34.267713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaa888-8f34-4d8e-bcaf-fa7ce61c78fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:34.268381 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaa888-8f34-4d8e-bcaf-fa7ce61c78fb None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:34.268702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaa888-8f34-4d8e-bcaf-fa7ce61c78fb None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:34.268980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaa888-8f34-4d8e-bcaf-fa7ce61c78fb None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:34.269735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efaa888-8f34-4d8e-bcaf-fa7ce61c78fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:34.270028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efaa888-8f34-4d8e-bcaf-fa7ce61c78fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:34.271411 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efaa888-8f34-4d8e-bcaf-fa7ce61c78fb None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:34.279615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 224/447] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:34.280477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaa888-8f34-4d8e-bcaf-fa7ce61c78fb None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:34.309565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4b5e6-9452-47ac-b7a9-b4d89ef0361c None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:34.341772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aec015-6735-4d28-884d-9997b79d713d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:34.342120 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:34.342636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aec015-6735-4d28-884d-9997b79d713d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:34.342923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aec015-6735-4d28-884d-9997b79d713d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:34.343279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aec015-6735-4d28-884d-9997b79d713d None admin] PATH_INFO: `/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:34.344845 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aec015-6735-4d28-884d-9997b79d713d None admin] RBAC: Authorizing `identity:get_project(project_id=2a11c903d8ba43eaa621b95be9cb82cb)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:34.346291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aec015-6735-4d28-884d-9997b79d713d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:34.347977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 224/448] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:34 2022] GET /identity/v3/projects/2a11c903d8ba43eaa621b95be9cb82cb => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:34.349248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aec015-6735-4d28-884d-9997b79d713d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:34.387830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaa888-8f34-4d8e-bcaf-fa7ce61c78fb None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:34.399415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:34.421446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4eddb-cbce-4286-a046-c5da6a33b932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:34.422108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4eddb-cbce-4286-a046-c5da6a33b932 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:34.422394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4eddb-cbce-4286-a046-c5da6a33b932 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:34.422670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4eddb-cbce-4286-a046-c5da6a33b932 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:34.423503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4eddb-cbce-4286-a046-c5da6a33b932 None admin] RBAC: Adding query filter params (group.id=cf32feb4691047099d7e3ea98965d01f, role.id=82ee29c384454b9aa4db085c9d2e21e8, scope.project.id=d68d17929cf4421f977087a339234c2a) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:34.423811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4eddb-cbce-4286-a046-c5da6a33b932 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:34.425276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4eddb-cbce-4286-a046-c5da6a33b932 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:34.431650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 225/449] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Jul 25 08:32:34 2022] GET /identity/v3/role_assignments?group.id=cf32feb4691047099d7e3ea98965d01f&scope.project.id=d68d17929cf4421f977087a339234c2a&role.id=82ee29c384454b9aa4db085c9d2e21e8 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:34.432427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4eddb-cbce-4286-a046-c5da6a33b932 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:34.458720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aec015-6735-4d28-884d-9997b79d713d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:34.499027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251cea1-40fe-4f50-af73-8ba99f6e4b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:34.499704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251cea1-40fe-4f50-af73-8ba99f6e4b94 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:34.499992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251cea1-40fe-4f50-af73-8ba99f6e4b94 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:34.500563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251cea1-40fe-4f50-af73-8ba99f6e4b94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:34.546230 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4eddb-cbce-4286-a046-c5da6a33b932 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:34.574066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:34.579221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251cea1-40fe-4f50-af73-8ba99f6e4b94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:34.580732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251cea1-40fe-4f50-af73-8ba99f6e4b94 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:34.584062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa935e81-e079-4e5c-99fc-1ebd71033d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:34.584958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa935e81-e079-4e5c-99fc-1ebd71033d88 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:34.585245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa935e81-e079-4e5c-99fc-1ebd71033d88 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:34.585522 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa935e81-e079-4e5c-99fc-1ebd71033d88 None admin] PATH_INFO: `/v3/groups/cf32feb4691047099d7e3ea98965d01f` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:34.586719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa935e81-e079-4e5c-99fc-1ebd71033d88 None admin] RBAC: Authorizing `identity:get_group(group_id=cf32feb4691047099d7e3ea98965d01f)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:34.588149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa935e81-e079-4e5c-99fc-1ebd71033d88 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:34.590108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 226/450] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:34 2022] GET /identity/v3/groups/cf32feb4691047099d7e3ea98965d01f => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:34.595519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa935e81-e079-4e5c-99fc-1ebd71033d88 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:34.644919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:34.702074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 225/451] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:34.707721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251cea1-40fe-4f50-af73-8ba99f6e4b94 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:34.717136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa935e81-e079-4e5c-99fc-1ebd71033d88 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:34.762645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f4caf-c0a2-4ddc-9e95-b109d753f351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:34.763266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f4caf-c0a2-4ddc-9e95-b109d753f351 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:34.763552 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f4caf-c0a2-4ddc-9e95-b109d753f351 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:34.763829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f4caf-c0a2-4ddc-9e95-b109d753f351 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:34.766824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f4caf-c0a2-4ddc-9e95-b109d753f351 None admin] RBAC: Adding query filter params (group.id=cf32feb4691047099d7e3ea98965d01f, role.id=3ce6c4ec0ed142828cb20882e342328e, scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:34.766824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f4caf-c0a2-4ddc-9e95-b109d753f351 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:34.766824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f4caf-c0a2-4ddc-9e95-b109d753f351 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:34.773132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 227/452] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Jul 25 08:32:34 2022] GET /identity/v3/role_assignments?group.id=cf32feb4691047099d7e3ea98965d01f&scope.project.id=2a11c903d8ba43eaa621b95be9cb82cb&role.id=3ce6c4ec0ed142828cb20882e342328e => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:34.820311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251cea1-40fe-4f50-af73-8ba99f6e4b94 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:34.859938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf89f6-1243-4ed6-90cf-5629d6a01414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:34.861161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf89f6-1243-4ed6-90cf-5629d6a01414 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:34.861638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf89f6-1243-4ed6-90cf-5629d6a01414 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:34.861914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf89f6-1243-4ed6-90cf-5629d6a01414 None admin] PATH_INFO: `/v3/projects/e86d2d432f134fa4aeaa99c3935e4c0c` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:34.863207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf89f6-1243-4ed6-90cf-5629d6a01414 None admin] RBAC: Authorizing `identity:get_project(project_id=e86d2d432f134fa4aeaa99c3935e4c0c)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:34.864747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf89f6-1243-4ed6-90cf-5629d6a01414 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:34.866675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 226/453] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:32:34 2022] GET /identity/v3/projects/e86d2d432f134fa4aeaa99c3935e4c0c => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:34.872370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f4caf-c0a2-4ddc-9e95-b109d753f351 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:34.945021 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:34.990325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f4caf-c0a2-4ddc-9e95-b109d753f351 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:35.026221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fc4d1-52af-42b6-a6d3-e4297c78c9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:35.027085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fc4d1-52af-42b6-a6d3-e4297c78c9e5 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:35.027368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fc4d1-52af-42b6-a6d3-e4297c78c9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:35.027644 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fc4d1-52af-42b6-a6d3-e4297c78c9e5 None admin] PATH_INFO: `/v3/groups/5860b923e33c461f87ef80dfeb37b59d` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:35.028840 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fc4d1-52af-42b6-a6d3-e4297c78c9e5 None admin] RBAC: Authorizing `identity:get_group(group_id=5860b923e33c461f87ef80dfeb37b59d)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:35.030530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fc4d1-52af-42b6-a6d3-e4297c78c9e5 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:35.032151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 228/454] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Jul 25 08:32:34 2022] GET /identity/v3/groups/5860b923e33c461f87ef80dfeb37b59d => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:35.038191 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf89f6-1243-4ed6-90cf-5629d6a01414 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:35.061277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:35.142021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf89f6-1243-4ed6-90cf-5629d6a01414 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:35.176524 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95b7ec-1011-4cca-a609-b8249c159e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:35.177246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95b7ec-1011-4cca-a609-b8249c159e0b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:35.177527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95b7ec-1011-4cca-a609-b8249c159e0b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:35.177808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95b7ec-1011-4cca-a609-b8249c159e0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:35.178654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95b7ec-1011-4cca-a609-b8249c159e0b None admin] RBAC: Adding query filter params (group.id=5860b923e33c461f87ef80dfeb37b59d, role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=e86d2d432f134fa4aeaa99c3935e4c0c) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:35.178968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95b7ec-1011-4cca-a609-b8249c159e0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:35.180403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95b7ec-1011-4cca-a609-b8249c159e0b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:35.186346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 227/455] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Jul 25 08:32:35 2022] GET /identity/v3/role_assignments?group.id=5860b923e33c461f87ef80dfeb37b59d&scope.project.id=e86d2d432f134fa4aeaa99c3935e4c0c&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:37.565123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9043951-29a1-4b76-be83-1abf12b49c19 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:37.566192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9043951-29a1-4b76-be83-1abf12b49c19 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:37.566691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9043951-29a1-4b76-be83-1abf12b49c19 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:37.568590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 229/456] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:37.574009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20babab2-3328-4400-a001-3e970fc317b7 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:37.574808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20babab2-3328-4400-a001-3e970fc317b7 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:37.575294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20babab2-3328-4400-a001-3e970fc317b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:37.601503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-20babab2-3328-4400-a001-3e970fc317b7 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:37.603005 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-20babab2-3328-4400-a001-3e970fc317b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:37.687372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 228/457] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:37.732921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9043951-29a1-4b76-be83-1abf12b49c19 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:37.873355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9043951-29a1-4b76-be83-1abf12b49c19 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:37.910618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:37.921051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d671f6-c95b-461f-819c-3a77fd6f4bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:37.921855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d671f6-c95b-461f-819c-3a77fd6f4bed None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:37.922141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d671f6-c95b-461f-819c-3a77fd6f4bed None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:37.922417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d671f6-c95b-461f-819c-3a77fd6f4bed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:37.935273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d671f6-c95b-461f-819c-3a77fd6f4bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:37.936763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d671f6-c95b-461f-819c-3a77fd6f4bed None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-32d671f6-c95b-461f-819c-3a77fd6f4bed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:37.959883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:37.960804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:37.961633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:37.961633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:37.961633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:37.961633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:37.965384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 230/458] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:37.971470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20babab2-3328-4400-a001-3e970fc317b7 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:37.988386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb97af-1d43-4d4a-b17e-0c496ddfb223 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:37.989098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb97af-1d43-4d4a-b17e-0c496ddfb223 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:37.989596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb97af-1d43-4d4a-b17e-0c496ddfb223 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:37.989698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:37.991515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 231/459] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:37.997279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f812033-6ddb-45ad-8392-3b2e9b31b3ac None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:37.997808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f812033-6ddb-45ad-8392-3b2e9b31b3ac None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:37.998276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f812033-6ddb-45ad-8392-3b2e9b31b3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.023828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-8f812033-6ddb-45ad-8392-3b2e9b31b3ac None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:38.025076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8f812033-6ddb-45ad-8392-3b2e9b31b3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:38.096423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 232/460] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:38.097808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745df4b-6511-4e1e-8420-1cd8b37d1be6 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:38.098098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745df4b-6511-4e1e-8420-1cd8b37d1be6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:38.098380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745df4b-6511-4e1e-8420-1cd8b37d1be6 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.098611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20babab2-3328-4400-a001-3e970fc317b7 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:38.099939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 233/461] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:38.111839 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3602184-1344-47c9-87af-59bec2df6329 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:38.112133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3602184-1344-47c9-87af-59bec2df6329 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:38.112522 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3602184-1344-47c9-87af-59bec2df6329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.137615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d3602184-1344-47c9-87af-59bec2df6329 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:38.138591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dddba-4989-4ec7-80fa-d462c28d3d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:38.138818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d3602184-1344-47c9-87af-59bec2df6329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:38.139263 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dddba-4989-4ec7-80fa-d462c28d3d31 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:38.139549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dddba-4989-4ec7-80fa-d462c28d3d31 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:38.139825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dddba-4989-4ec7-80fa-d462c28d3d31 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.140598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4dddba-4989-4ec7-80fa-d462c28d3d31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:38.140985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4dddba-4989-4ec7-80fa-d462c28d3d31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:38.142578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4dddba-4989-4ec7-80fa-d462c28d3d31 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:38.162200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 229/462] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:38.163075 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dddba-4989-4ec7-80fa-d462c28d3d31 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:38.212221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 234/463] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:38.212869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3602184-1344-47c9-87af-59bec2df6329 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:38.253961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:38.268136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:38.310623 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dddba-4989-4ec7-80fa-d462c28d3d31 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:38.320287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3602184-1344-47c9-87af-59bec2df6329 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:38.347015 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce2a3c-2827-489a-b28c-679d67dce432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:38.347891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce2a3c-2827-489a-b28c-679d67dce432 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:38.348101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce2a3c-2827-489a-b28c-679d67dce432 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:38.348436 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce2a3c-2827-489a-b28c-679d67dce432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.358648 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fec29-d8b8-469a-87e1-9e2a41f5d5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:38.359512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fec29-d8b8-469a-87e1-9e2a41f5d5a0 None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:38.359744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce2a3c-2827-489a-b28c-679d67dce432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:38.360115 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fec29-d8b8-469a-87e1-9e2a41f5d5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:38.360535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fec29-d8b8-469a-87e1-9e2a41f5d5a0 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.361361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce2a3c-2827-489a-b28c-679d67dce432 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:38.361644 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fec29-d8b8-469a-87e1-9e2a41f5d5a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:38.363317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fec29-d8b8-469a-87e1-9e2a41f5d5a0 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-13ce2a3c-2827-489a-b28c-679d67dce432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:38.382392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:38.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:38.384221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:38.384221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:38.384221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:38.384221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:38.384221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 230/464] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:38.384475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce2a3c-2827-489a-b28c-679d67dce432 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:38.391887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 235/465] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:32:38 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:32:38.393597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5706566-2d48-4450-ac38-66c6ef69b465 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:38.394115 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5706566-2d48-4450-ac38-66c6ef69b465 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:38.394752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5706566-2d48-4450-ac38-66c6ef69b465 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.401181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 236/466] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:38.402599 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5706566-2d48-4450-ac38-66c6ef69b465 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:38.526993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce2a3c-2827-489a-b28c-679d67dce432 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:38.532287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:38.544930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5706566-2d48-4450-ac38-66c6ef69b465 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:38.561275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:38.572436 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ea45c-c471-40c8-b4e1-65ce09a5c768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:38.573284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ea45c-c471-40c8-b4e1-65ce09a5c768 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:38.573567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ea45c-c471-40c8-b4e1-65ce09a5c768 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:38.573843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ea45c-c471-40c8-b4e1-65ce09a5c768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.587210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ea45c-c471-40c8-b4e1-65ce09a5c768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:38.589451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ea45c-c471-40c8-b4e1-65ce09a5c768 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:38.595011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145ce30-9103-4360-8202-39cb097e49dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:38.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145ce30-9103-4360-8202-39cb097e49dc None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:38.595972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145ce30-9103-4360-8202-39cb097e49dc None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:38.596363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145ce30-9103-4360-8202-39cb097e49dc None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.597171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145ce30-9103-4360-8202-39cb097e49dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:38.597476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145ce30-9103-4360-8202-39cb097e49dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:38.598878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145ce30-9103-4360-8202-39cb097e49dc None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-b67ea45c-c471-40c8-b4e1-65ce09a5c768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:38.607733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:38.608598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:38.609793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:38.609793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:38.609793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:38.609793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:38.609793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 231/467] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:38.610061 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae45e29-1b5e-4a06-84f9-5a74d306bc2b None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:38.610774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae45e29-1b5e-4a06-84f9-5a74d306bc2b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:38.610774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae45e29-1b5e-4a06-84f9-5a74d306bc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.619816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 237/468] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:38.621614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145ce30-9103-4360-8202-39cb097e49dc None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:38.634948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-fae45e29-1b5e-4a06-84f9-5a74d306bc2b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:38.636312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-fae45e29-1b5e-4a06-84f9-5a74d306bc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:38.713889 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 232/469] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:38.714647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae45e29-1b5e-4a06-84f9-5a74d306bc2b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:38.773983 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145ce30-9103-4360-8202-39cb097e49dc None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:38.810938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4602c47-1760-4726-b5ef-92015f43fe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:38.811610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602c47-1760-4726-b5ef-92015f43fe63 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:38.811892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602c47-1760-4726-b5ef-92015f43fe63 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:38.812169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602c47-1760-4726-b5ef-92015f43fe63 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.812926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4602c47-1760-4726-b5ef-92015f43fe63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:38.813233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4602c47-1760-4726-b5ef-92015f43fe63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:38.814607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4602c47-1760-4726-b5ef-92015f43fe63 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:38.827582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 238/470] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:38.828595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4602c47-1760-4726-b5ef-92015f43fe63 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:38.833815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae45e29-1b5e-4a06-84f9-5a74d306bc2b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:38.843446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:38.844286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:38.870947 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fae768-169c-4d90-bb88-5a4f14e3c9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:38.871624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fae768-169c-4d90-bb88-5a4f14e3c9d4 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:38.871905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fae768-169c-4d90-bb88-5a4f14e3c9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:38.872180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fae768-169c-4d90-bb88-5a4f14e3c9d4 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.872999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fae768-169c-4d90-bb88-5a4f14e3c9d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:38.874512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fae768-169c-4d90-bb88-5a4f14e3c9d4 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:38.899451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 233/471] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:32:38 2022] POST /identity/v3/users => generated 248 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:32:38.900335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fae768-169c-4d90-bb88-5a4f14e3c9d4 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:38.941289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4602c47-1760-4726-b5ef-92015f43fe63 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:38.979516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16713f-9d14-47c6-b318-54eeef5a1fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:38.980356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16713f-9d14-47c6-b318-54eeef5a1fc8 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:38.980681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16713f-9d14-47c6-b318-54eeef5a1fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:38.980961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16713f-9d14-47c6-b318-54eeef5a1fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:38.993624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16713f-9d14-47c6-b318-54eeef5a1fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:38.995201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16713f-9d14-47c6-b318-54eeef5a1fc8 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-ef16713f-9d14-47c6-b318-54eeef5a1fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:39.014833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:39.015757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:39.017417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:39.017417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:39.017417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:39.017417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:39.017417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 239/472] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:39.018101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583d936-ba3d-4b9f-b28d-7131236c2c00 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:39.018101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583d936-ba3d-4b9f-b28d-7131236c2c00 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:39.018372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583d936-ba3d-4b9f-b28d-7131236c2c00 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:39.021684 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 240/473] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:39.023290 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fae768-169c-4d90-bb88-5a4f14e3c9d4 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:39.023469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583d936-ba3d-4b9f-b28d-7131236c2c00 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:39.062371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19c4c2-44d6-44fe-bc77-f71b13ece7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:39.063055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19c4c2-44d6-44fe-bc77-f71b13ece7eb None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:39.063339 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19c4c2-44d6-44fe-bc77-f71b13ece7eb None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:39.063616 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19c4c2-44d6-44fe-bc77-f71b13ece7eb None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:39.064334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19c4c2-44d6-44fe-bc77-f71b13ece7eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:39.066554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19c4c2-44d6-44fe-bc77-f71b13ece7eb None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:39.091517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 234/474] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:32:38 2022] POST /identity/v3/users => generated 248 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:32:39.092592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7738e3-7c92-4159-991f-62f45c7271df None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:39.092885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7738e3-7c92-4159-991f-62f45c7271df None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:39.093163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7738e3-7c92-4159-991f-62f45c7271df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:39.105398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:39.115271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ad7738e3-7c92-4159-991f-62f45c7271df None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:39.116514 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ad7738e3-7c92-4159-991f-62f45c7271df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:39.135481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:39.152536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583d936-ba3d-4b9f-b28d-7131236c2c00 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:39.200628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181469cd-95b4-47c3-852b-e1ea82bd7948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:39.201726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181469cd-95b4-47c3-852b-e1ea82bd7948 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:39.202212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181469cd-95b4-47c3-852b-e1ea82bd7948 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:39.202783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181469cd-95b4-47c3-852b-e1ea82bd7948 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:39.203740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181469cd-95b4-47c3-852b-e1ea82bd7948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:39.204243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 235/475] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:39.204420 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181469cd-95b4-47c3-852b-e1ea82bd7948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:39.209331 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181469cd-95b4-47c3-852b-e1ea82bd7948 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:39.225506 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 241/476] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:39.231508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7738e3-7c92-4159-991f-62f45c7271df None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:39.235491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181469cd-95b4-47c3-852b-e1ea82bd7948 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:39.373442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7738e3-7c92-4159-991f-62f45c7271df None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:39.385945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181469cd-95b4-47c3-852b-e1ea82bd7948 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:39.413953 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1cbfe-69b3-47f2-af59-2bf01f9153ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:39.414624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1cbfe-69b3-47f2-af59-2bf01f9153ae None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:39.414905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1cbfe-69b3-47f2-af59-2bf01f9153ae None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:39.415178 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1cbfe-69b3-47f2-af59-2bf01f9153ae None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:39.416731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1cbfe-69b3-47f2-af59-2bf01f9153ae None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:39.418164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1cbfe-69b3-47f2-af59-2bf01f9153ae None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:39.420396 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:39.429382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.sql.core [None req-bbe1cbfe-69b3-47f2-af59-2bf01f9153ae None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 08:32:39.429382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 08:32:39.429382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [parameters: {'id': 'eed88345b2394ffcb8cca06387efaf9e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 08:32:39.429382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86751) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 08:32:39.433347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d6cd2-deb8-4cfe-b778-c93710edc6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:39.434183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d6cd2-deb8-4cfe-b778-c93710edc6ba None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:39.434678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d6cd2-deb8-4cfe-b778-c93710edc6ba None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:39.435095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d6cd2-deb8-4cfe-b778-c93710edc6ba None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:39.435958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d6cd2-deb8-4cfe-b778-c93710edc6ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:39.437610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d6cd2-deb8-4cfe-b778-c93710edc6ba None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-bbe1cbfe-69b3-47f2-af59-2bf01f9153ae None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application conn.query(q) Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application result.read() Jul 25 08:32:39.450155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return ref.to_dict() Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 08:32:39.451876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application next(self.gen) Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application yield resource Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application next(self.gen) Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise self.value Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application session.commit() Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 08:32:39.453037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self.session.flush() Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application compat.raise_( Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 08:32:39.454030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 08:32:39.455121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application conn.query(q) Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application result.read() Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 08:32:39.456086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [parameters: {'id': 'eed88345b2394ffcb8cca06387efaf9e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:39.457938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 08:32:39.459816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:39.461564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 236/477] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Jul 25 08:32:39 2022] POST /identity/v3/roles => generated 144 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:39.461564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1cbfe-69b3-47f2-af59-2bf01f9153ae None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:39.472142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 242/478] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:32:39 2022] POST /identity/v3/users => generated 249 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:32:39.494729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:39.601465 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1cbfe-69b3-47f2-af59-2bf01f9153ae None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:39.650816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05fcc3-179b-4084-a1d4-03bff0bb35b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:39.651652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05fcc3-179b-4084-a1d4-03bff0bb35b4 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:39.651938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05fcc3-179b-4084-a1d4-03bff0bb35b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:39.652214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05fcc3-179b-4084-a1d4-03bff0bb35b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:39.673603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05fcc3-179b-4084-a1d4-03bff0bb35b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-7f05fcc3-179b-4084-a1d4-03bff0bb35b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:39.682541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:39.683481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:39.684273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 237/479] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:32:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:39.689595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d6cd2-deb8-4cfe-b778-c93710edc6ba None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:39.805638 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d6cd2-deb8-4cfe-b778-c93710edc6ba None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:39.842939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc91483-8083-4080-99da-d726c41196b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:39.843616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc91483-8083-4080-99da-d726c41196b4 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:39.843954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc91483-8083-4080-99da-d726c41196b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:39.844239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc91483-8083-4080-99da-d726c41196b4 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:39.845047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc91483-8083-4080-99da-d726c41196b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:39.845453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc91483-8083-4080-99da-d726c41196b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:39.846832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc91483-8083-4080-99da-d726c41196b4 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:39.856067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 243/480] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:40.754176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b920-6c13-46a5-938d-93e004b7b575 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:40.755503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b920-6c13-46a5-938d-93e004b7b575 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:40.755503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b920-6c13-46a5-938d-93e004b7b575 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:40.756943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 238/481] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:40.762347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b0f00-bb16-48be-ab69-1992add2411c None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:40.762845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b0f00-bb16-48be-ab69-1992add2411c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:40.763031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b0f00-bb16-48be-ab69-1992add2411c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:40.785513 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-eb5b0f00-bb16-48be-ab69-1992add2411c None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:40.786761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-eb5b0f00-bb16-48be-ab69-1992add2411c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:40.834390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:40.874900 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 244/482] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:40.914589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9b920-6c13-46a5-938d-93e004b7b575 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:41.023091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:41.062215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9b920-6c13-46a5-938d-93e004b7b575 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:41.110969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88fac6-b39c-4920-9c63-a9301b47dd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:41.111767 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88fac6-b39c-4920-9c63-a9301b47dd0a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:41.112054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88fac6-b39c-4920-9c63-a9301b47dd0a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:41.112331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88fac6-b39c-4920-9c63-a9301b47dd0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:41.128420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88fac6-b39c-4920-9c63-a9301b47dd0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-3b88fac6-b39c-4920-9c63-a9301b47dd0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:41.136148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:41.137145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:41.138532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 239/483] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:41.143173 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b0f00-bb16-48be-ab69-1992add2411c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:41.258732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b0f00-bb16-48be-ab69-1992add2411c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:41.311194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2c92a-3615-49ab-8960-3e31af34fdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:41.312056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2c92a-3615-49ab-8960-3e31af34fdf1 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:41.312506 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2c92a-3615-49ab-8960-3e31af34fdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:41.312943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2c92a-3615-49ab-8960-3e31af34fdf1 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:41.313846 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2c92a-3615-49ab-8960-3e31af34fdf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:41.314289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2c92a-3615-49ab-8960-3e31af34fdf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:41.315797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2c92a-3615-49ab-8960-3e31af34fdf1 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:41.325275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 245/484] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:41.330855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88fac6-b39c-4920-9c63-a9301b47dd0a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:41.458583 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88fac6-b39c-4920-9c63-a9301b47dd0a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:41.480819 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f041b-13fa-4f84-8325-95de29cc0ff6 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:41.481109 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f041b-13fa-4f84-8325-95de29cc0ff6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:41.481422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f041b-13fa-4f84-8325-95de29cc0ff6 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:41.483055 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 246/485] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:41.487797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53659cb4-b1f1-422c-885c-9bc4d926e7d3 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:41.488095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53659cb4-b1f1-422c-885c-9bc4d926e7d3 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:41.488384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53659cb4-b1f1-422c-885c-9bc4d926e7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:41.493001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f276d-e0b4-4c0c-afa8-99ca5eb09c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:41.493799 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f276d-e0b4-4c0c-afa8-99ca5eb09c6b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:41.494083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f276d-e0b4-4c0c-afa8-99ca5eb09c6b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:41.494362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f276d-e0b4-4c0c-afa8-99ca5eb09c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:41.497373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:41.511662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f276d-e0b4-4c0c-afa8-99ca5eb09c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:41.513375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f276d-e0b4-4c0c-afa8-99ca5eb09c6b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:41.521233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-53659cb4-b1f1-422c-885c-9bc4d926e7d3 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:41.522602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-53659cb4-b1f1-422c-885c-9bc4d926e7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-850f276d-e0b4-4c0c-afa8-99ca5eb09c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:41.534750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:41.535837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:41.536931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:41.536931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:41.536931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:41.536931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:41.538850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 240/486] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:41.548767 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f276d-e0b4-4c0c-afa8-99ca5eb09c6b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:41.592419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:41.656392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f276d-e0b4-4c0c-afa8-99ca5eb09c6b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:41.672614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 247/487] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:32:41.694659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53659cb4-b1f1-422c-885c-9bc4d926e7d3 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:41.705284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46e397-e493-40e1-a5da-04e98839d4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:41.705966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46e397-e493-40e1-a5da-04e98839d4a6 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:41.706249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46e397-e493-40e1-a5da-04e98839d4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:41.706528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46e397-e493-40e1-a5da-04e98839d4a6 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:41.707270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46e397-e493-40e1-a5da-04e98839d4a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:41.707614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46e397-e493-40e1-a5da-04e98839d4a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:41.709058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46e397-e493-40e1-a5da-04e98839d4a6 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:41.737122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 241/488] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:41.743469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46e397-e493-40e1-a5da-04e98839d4a6 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:41.834045 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:41.884082 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53659cb4-b1f1-422c-885c-9bc4d926e7d3 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:41.929567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46e397-e493-40e1-a5da-04e98839d4a6 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:41.946080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d468f3-a264-4182-94be-b6a3fd1b7b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:41.946895 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d468f3-a264-4182-94be-b6a3fd1b7b95 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:41.947181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d468f3-a264-4182-94be-b6a3fd1b7b95 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:41.947461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d468f3-a264-4182-94be-b6a3fd1b7b95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:41.969851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d468f3-a264-4182-94be-b6a3fd1b7b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-f3d468f3-a264-4182-94be-b6a3fd1b7b95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:41.978304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:41.979154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:41.979934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:41.979934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:41.979934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:41.979934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:41.979934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:41.979934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:41.979934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:41.979934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:41.979934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:41.979934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:41.979934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:41.979934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:41.985995 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 248/489] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:41.992863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d468f3-a264-4182-94be-b6a3fd1b7b95 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:42.000681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd472b33-c6f5-4fcd-a510-0a7ac9fb126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:42.001501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd472b33-c6f5-4fcd-a510-0a7ac9fb126e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.001789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd472b33-c6f5-4fcd-a510-0a7ac9fb126e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.002064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd472b33-c6f5-4fcd-a510-0a7ac9fb126e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.015795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd472b33-c6f5-4fcd-a510-0a7ac9fb126e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:42.017302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd472b33-c6f5-4fcd-a510-0a7ac9fb126e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-bd472b33-c6f5-4fcd-a510-0a7ac9fb126e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:42.037682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:42.038617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:42.039478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:42.039478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:42.039478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:42.039478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:42.039478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:42.039478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:42.039478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:42.039478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:42.039478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:32:42.039478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:42.039478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 242/490] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Jul 25 08:32:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:42.040511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:42.040629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9f505-2c52-4156-a00c-486ea3eabb6b None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.040788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9f505-2c52-4156-a00c-486ea3eabb6b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.044591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9f505-2c52-4156-a00c-486ea3eabb6b None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.046350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 243/491] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:42.047160 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9f505-2c52-4156-a00c-486ea3eabb6b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:42.122026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d468f3-a264-4182-94be-b6a3fd1b7b95 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:42.156029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9f505-2c52-4156-a00c-486ea3eabb6b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:42.158297 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adf400-6a8c-4a2c-a5ce-cdfe3d5483c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:42.159083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adf400-6a8c-4a2c-a5ce-cdfe3d5483c3 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.159373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adf400-6a8c-4a2c-a5ce-cdfe3d5483c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.159655 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adf400-6a8c-4a2c-a5ce-cdfe3d5483c3 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.160423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79adf400-6a8c-4a2c-a5ce-cdfe3d5483c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:42.160803 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79adf400-6a8c-4a2c-a5ce-cdfe3d5483c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:42.162191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79adf400-6a8c-4a2c-a5ce-cdfe3d5483c3 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:42.164071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:42.171237 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 249/492] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:42.173170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e56173-beb5-4b9e-be85-08c1b24fbb22 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.173671 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e56173-beb5-4b9e-be85-08c1b24fbb22 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.174178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e56173-beb5-4b9e-be85-08c1b24fbb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.193746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-48e56173-beb5-4b9e-be85-08c1b24fbb22 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:42.195133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-48e56173-beb5-4b9e-be85-08c1b24fbb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:42.195318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9eb5fe-f317-415f-a613-373a33cf305b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:42.196066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9eb5fe-f317-415f-a613-373a33cf305b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.196357 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9eb5fe-f317-415f-a613-373a33cf305b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.196671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9eb5fe-f317-415f-a613-373a33cf305b None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.197622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9eb5fe-f317-415f-a613-373a33cf305b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:42.197836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9eb5fe-f317-415f-a613-373a33cf305b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:42.199532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9eb5fe-f317-415f-a613-373a33cf305b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:42.236907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 244/493] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Jul 25 08:32:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:42.238314 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f04000-dea2-4777-b3c4-f5da8bfe3478 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.238611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f04000-dea2-4777-b3c4-f5da8bfe3478 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.239419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f04000-dea2-4777-b3c4-f5da8bfe3478 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.240870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 245/494] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:42.245469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f04000-dea2-4777-b3c4-f5da8bfe3478 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:42.271510 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 250/495] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:42.272363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e56173-beb5-4b9e-be85-08c1b24fbb22 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:42.296207 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:42.364885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f04000-dea2-4777-b3c4-f5da8bfe3478 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:42.387781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e56173-beb5-4b9e-be85-08c1b24fbb22 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:42.399384 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57a9f8-6657-48cd-9e1f-4afe0225d002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:42.400153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57a9f8-6657-48cd-9e1f-4afe0225d002 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.400434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57a9f8-6657-48cd-9e1f-4afe0225d002 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.400745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57a9f8-6657-48cd-9e1f-4afe0225d002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.410815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57a9f8-6657-48cd-9e1f-4afe0225d002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:42.412215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57a9f8-6657-48cd-9e1f-4afe0225d002 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:42.421464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce14c79-2295-4d6f-a58f-056b035250de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:42.422118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce14c79-2295-4d6f-a58f-056b035250de None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.422403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce14c79-2295-4d6f-a58f-056b035250de None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.422682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce14c79-2295-4d6f-a58f-056b035250de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-ab57a9f8-6657-48cd-9e1f-4afe0225d002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:42.430107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:42.431062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:42.431888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:42.431888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:42.431888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:42.431888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:42.431888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 246/496] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:42.432875 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82140d-aa4f-45df-897f-cf7c9436e49a None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.432875 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82140d-aa4f-45df-897f-cf7c9436e49a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.433031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82140d-aa4f-45df-897f-cf7c9436e49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.452035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ae82140d-aa4f-45df-897f-cf7c9436e49a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:42.453246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ae82140d-aa4f-45df-897f-cf7c9436e49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:42.464513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:42.492354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce14c79-2295-4d6f-a58f-056b035250de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:42.493851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce14c79-2295-4d6f-a58f-056b035250de None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:42.511812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:42.519296 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 247/497] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:42.520136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82140d-aa4f-45df-897f-cf7c9436e49a None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:42.592290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 251/498] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:42.593636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af37c6-3508-453e-ad35-d356edee3cc9 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.593930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af37c6-3508-453e-ad35-d356edee3cc9 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.594212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af37c6-3508-453e-ad35-d356edee3cc9 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.595489 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 252/499] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:42.596181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af37c6-3508-453e-ad35-d356edee3cc9 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:42.622395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82140d-aa4f-45df-897f-cf7c9436e49a None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:42.654592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8726a-27fb-4268-8155-19cef4fc0771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:42.655385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8726a-27fb-4268-8155-19cef4fc0771 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.655668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8726a-27fb-4268-8155-19cef4fc0771 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.655937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8726a-27fb-4268-8155-19cef4fc0771 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.666610 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8726a-27fb-4268-8155-19cef4fc0771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-81d8726a-27fb-4268-8155-19cef4fc0771 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:42.674358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:42.675262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:42.676046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 248/500] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:42.677637 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8726a-27fb-4268-8155-19cef4fc0771 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:42.685966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:42.701508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af37c6-3508-453e-ad35-d356edee3cc9 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:42.719112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:42.737624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c6115-c769-40e8-8e23-8df79dd65947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:42.738289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c6115-c769-40e8-8e23-8df79dd65947 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.738577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c6115-c769-40e8-8e23-8df79dd65947 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.739421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c6115-c769-40e8-8e23-8df79dd65947 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.740148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c6115-c769-40e8-8e23-8df79dd65947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:42.740456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c6115-c769-40e8-8e23-8df79dd65947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:42.741862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c6115-c769-40e8-8e23-8df79dd65947 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:42.753140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 253/501] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:42.753922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c6115-c769-40e8-8e23-8df79dd65947 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:42.778364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8726a-27fb-4268-8155-19cef4fc0771 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:42.812499 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11739dbb-fa65-44be-9a3d-ff6ba55da81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:42.813624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11739dbb-fa65-44be-9a3d-ff6ba55da81c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.813910 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11739dbb-fa65-44be-9a3d-ff6ba55da81c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.814185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11739dbb-fa65-44be-9a3d-ff6ba55da81c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.824022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11739dbb-fa65-44be-9a3d-ff6ba55da81c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-11739dbb-fa65-44be-9a3d-ff6ba55da81c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:42.831632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:42.832497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:42.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 249/502] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:42.834012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f414d-9d8a-4180-bfc4-02295f64405b None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.834261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f414d-9d8a-4180-bfc4-02295f64405b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.834765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f414d-9d8a-4180-bfc4-02295f64405b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.852036 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-728f414d-9d8a-4180-bfc4-02295f64405b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:42.853231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-728f414d-9d8a-4180-bfc4-02295f64405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:42.857004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c6115-c769-40e8-8e23-8df79dd65947 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:42.890787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248a3c8-a623-4430-95f6-cb67e6c33370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:42.891574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248a3c8-a623-4430-95f6-cb67e6c33370 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:42.891858 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248a3c8-a623-4430-95f6-cb67e6c33370 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:42.892133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248a3c8-a623-4430-95f6-cb67e6c33370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:42.903263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248a3c8-a623-4430-95f6-cb67e6c33370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:42.905001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248a3c8-a623-4430-95f6-cb67e6c33370 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:42.912298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:42.920028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 250/503] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:42.920783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f414d-9d8a-4180-bfc4-02295f64405b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-e248a3c8-a623-4430-95f6-cb67e6c33370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:42.923122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:42.924023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:42.924896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:42.924896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:42.924896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:42.924896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:42.924896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 254/504] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:42.925148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248a3c8-a623-4430-95f6-cb67e6c33370 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:42.967530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:43.022548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f414d-9d8a-4180-bfc4-02295f64405b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.037271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248a3c8-a623-4430-95f6-cb67e6c33370 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.056167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887bcf6-1198-4b51-8f40-a809f2710e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.056877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887bcf6-1198-4b51-8f40-a809f2710e7f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.057162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887bcf6-1198-4b51-8f40-a809f2710e7f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.057439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887bcf6-1198-4b51-8f40-a809f2710e7f None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.058190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5887bcf6-1198-4b51-8f40-a809f2710e7f None admin] RBAC: Adding query filter params (name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:43.058497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5887bcf6-1198-4b51-8f40-a809f2710e7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.059893 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5887bcf6-1198-4b51-8f40-a809f2710e7f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.067625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 251/505] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:43.068400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887bcf6-1198-4b51-8f40-a809f2710e7f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:43.069746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477d808-9fff-482a-9b96-8f1e76dd90c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.070551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477d808-9fff-482a-9b96-8f1e76dd90c8 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.070827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477d808-9fff-482a-9b96-8f1e76dd90c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.071101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477d808-9fff-482a-9b96-8f1e76dd90c8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.079475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477d808-9fff-482a-9b96-8f1e76dd90c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.080923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477d808-9fff-482a-9b96-8f1e76dd90c8 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-1477d808-9fff-482a-9b96-8f1e76dd90c8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:43.096517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:43.097597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:43.099202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:43.099202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:43.099202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:43.099202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:43.099202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:43.099202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:43.099202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:43.099202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:43.099202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 08:32:43.099202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:43.099202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 255/506] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:43.099202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477d808-9fff-482a-9b96-8f1e76dd90c8 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:43.128911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:43.170186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887bcf6-1198-4b51-8f40-a809f2710e7f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.188494 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:43.198643 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477d808-9fff-482a-9b96-8f1e76dd90c8 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.202911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa17bb7-d6ca-4f9b-b86a-07aa77d31000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.203579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa17bb7-d6ca-4f9b-b86a-07aa77d31000 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.203865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa17bb7-d6ca-4f9b-b86a-07aa77d31000 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.204144 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa17bb7-d6ca-4f9b-b86a-07aa77d31000 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.204933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa17bb7-d6ca-4f9b-b86a-07aa77d31000 None admin] RBAC: Adding query filter params (name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:43.205244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa17bb7-d6ca-4f9b-b86a-07aa77d31000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.206628 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa17bb7-d6ca-4f9b-b86a-07aa77d31000 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.215109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 252/507] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:43.215891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa17bb7-d6ca-4f9b-b86a-07aa77d31000 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:43.233190 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1118933-5373-463f-825e-e3dabc82be7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.233857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1118933-5373-463f-825e-e3dabc82be7b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.234144 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1118933-5373-463f-825e-e3dabc82be7b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.234432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1118933-5373-463f-825e-e3dabc82be7b None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.235160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1118933-5373-463f-825e-e3dabc82be7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:43.235466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1118933-5373-463f-825e-e3dabc82be7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.236977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1118933-5373-463f-825e-e3dabc82be7b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.249192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 256/508] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:43.249973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1118933-5373-463f-825e-e3dabc82be7b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:43.318684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa17bb7-d6ca-4f9b-b86a-07aa77d31000 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.337664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:43.351992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e94b82-c7be-43f6-9f0c-c30e5bb192df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.352815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e94b82-c7be-43f6-9f0c-c30e5bb192df None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.353103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e94b82-c7be-43f6-9f0c-c30e5bb192df None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.353388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e94b82-c7be-43f6-9f0c-c30e5bb192df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.356826 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1118933-5373-463f-825e-e3dabc82be7b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.363663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e94b82-c7be-43f6-9f0c-c30e5bb192df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.365682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e94b82-c7be-43f6-9f0c-c30e5bb192df None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-39e94b82-c7be-43f6-9f0c-c30e5bb192df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:43.382452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:43.383446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:43.384334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:43.384334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:43.384334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:43.384334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:43.384334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 253/509] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:43.384629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e94b82-c7be-43f6-9f0c-c30e5bb192df None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:43.395725 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e383ad-a35e-4a0c-b663-e879f3a2555d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.396519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e383ad-a35e-4a0c-b663-e879f3a2555d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.396833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e383ad-a35e-4a0c-b663-e879f3a2555d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.397111 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e383ad-a35e-4a0c-b663-e879f3a2555d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.407312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e383ad-a35e-4a0c-b663-e879f3a2555d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.408771 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e383ad-a35e-4a0c-b663-e879f3a2555d None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-87e383ad-a35e-4a0c-b663-e879f3a2555d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:43.426390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:43.427770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:43.428636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:43.428636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:43.428636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:43.428636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:43.428636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 257/510] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:43.428893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e383ad-a35e-4a0c-b663-e879f3a2555d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:43.434683 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:43.483489 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e94b82-c7be-43f6-9f0c-c30e5bb192df None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.515881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aeddd8-1b57-4b11-a016-0680025b2ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.516541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aeddd8-1b57-4b11-a016-0680025b2ef2 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.516852 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aeddd8-1b57-4b11-a016-0680025b2ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.517128 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aeddd8-1b57-4b11-a016-0680025b2ef2 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.517957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aeddd8-1b57-4b11-a016-0680025b2ef2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:43.518264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aeddd8-1b57-4b11-a016-0680025b2ef2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.519686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aeddd8-1b57-4b11-a016-0680025b2ef2 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.530053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e383ad-a35e-4a0c-b663-e879f3a2555d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.536768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 254/511] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Jul 25 08:32:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:43.537570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aeddd8-1b57-4b11-a016-0680025b2ef2 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:43.564749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695b28f-34ca-4754-a441-b032f21bf4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.565533 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695b28f-34ca-4754-a441-b032f21bf4d9 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.565813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695b28f-34ca-4754-a441-b032f21bf4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.566092 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695b28f-34ca-4754-a441-b032f21bf4d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.566282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:43.576696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695b28f-34ca-4754-a441-b032f21bf4d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.578121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695b28f-34ca-4754-a441-b032f21bf4d9 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-1695b28f-34ca-4754-a441-b032f21bf4d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:43.594661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:43.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:43.596665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:43.596665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:43.596665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:43.596665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:43.596665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 258/512] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:43.596665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695b28f-34ca-4754-a441-b032f21bf4d9 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:43.636391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aeddd8-1b57-4b11-a016-0680025b2ef2 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.656387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:43.671414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353e88c-69c7-4e02-b144-45a0cb9b56c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.672205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353e88c-69c7-4e02-b144-45a0cb9b56c3 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.672492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353e88c-69c7-4e02-b144-45a0cb9b56c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.672815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353e88c-69c7-4e02-b144-45a0cb9b56c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.679622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353e88c-69c7-4e02-b144-45a0cb9b56c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.681046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353e88c-69c7-4e02-b144-45a0cb9b56c3 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-0353e88c-69c7-4e02-b144-45a0cb9b56c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:43.695741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:43.696754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:43.698132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:43.698132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:43.698132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:43.698132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:43.698132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:43.698132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:43.698132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:43.698132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:43.698132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 255/513] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:43.698132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353e88c-69c7-4e02-b144-45a0cb9b56c3 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:43.705541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695b28f-34ca-4754-a441-b032f21bf4d9 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509aa5b-45ed-4218-8088-67325656d325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.740502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509aa5b-45ed-4218-8088-67325656d325 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.740817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509aa5b-45ed-4218-8088-67325656d325 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.741097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509aa5b-45ed-4218-8088-67325656d325 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.741824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509aa5b-45ed-4218-8088-67325656d325 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:43.742131 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509aa5b-45ed-4218-8088-67325656d325 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.743501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509aa5b-45ed-4218-8088-67325656d325 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.755334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 259/514] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:43.756251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509aa5b-45ed-4218-8088-67325656d325 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:43.787688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:43.797405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353e88c-69c7-4e02-b144-45a0cb9b56c3 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.834740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5211971-3517-44ea-a074-c29b9f64a0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.835406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5211971-3517-44ea-a074-c29b9f64a0ad None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.835690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5211971-3517-44ea-a074-c29b9f64a0ad None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.835968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5211971-3517-44ea-a074-c29b9f64a0ad None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.836722 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5211971-3517-44ea-a074-c29b9f64a0ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:43.837032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5211971-3517-44ea-a074-c29b9f64a0ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.838416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5211971-3517-44ea-a074-c29b9f64a0ad None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.850097 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 256/515] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:43.850886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5211971-3517-44ea-a074-c29b9f64a0ad None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:43.856381 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509aa5b-45ed-4218-8088-67325656d325 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.870201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:43.890546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde96900-a6e8-4c78-a1f2-688c67c9342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.891188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde96900-a6e8-4c78-a1f2-688c67c9342d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.891703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde96900-a6e8-4c78-a1f2-688c67c9342d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.891799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde96900-a6e8-4c78-a1f2-688c67c9342d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.949105 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5211971-3517-44ea-a074-c29b9f64a0ad None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:43.956632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde96900-a6e8-4c78-a1f2-688c67c9342d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.958611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde96900-a6e8-4c78-a1f2-688c67c9342d None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.982675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89040f06-52e3-4f68-b4da-c95a878e4b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:43.983330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89040f06-52e3-4f68-b4da-c95a878e4b4a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:43.983613 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89040f06-52e3-4f68-b4da-c95a878e4b4a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:43.983891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89040f06-52e3-4f68-b4da-c95a878e4b4a None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:43.984646 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89040f06-52e3-4f68-b4da-c95a878e4b4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:43.984956 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89040f06-52e3-4f68-b4da-c95a878e4b4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:43.986332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89040f06-52e3-4f68-b4da-c95a878e4b4a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:43.997673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 257/516] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:43.998457 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89040f06-52e3-4f68-b4da-c95a878e4b4a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:44.011639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:44.058015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 260/517] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:44.058914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde96900-a6e8-4c78-a1f2-688c67c9342d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:44.075257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:44.099137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89040f06-52e3-4f68-b4da-c95a878e4b4a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:44.133616 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5322dd-9a0f-4300-ace1-f2995baade0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:44.134274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5322dd-9a0f-4300-ace1-f2995baade0b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:44.134561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5322dd-9a0f-4300-ace1-f2995baade0b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:44.134841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5322dd-9a0f-4300-ace1-f2995baade0b None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:44.135625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5322dd-9a0f-4300-ace1-f2995baade0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:44.135935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5322dd-9a0f-4300-ace1-f2995baade0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:44.137398 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5322dd-9a0f-4300-ace1-f2995baade0b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:44.155651 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 258/518] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:44.156439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5322dd-9a0f-4300-ace1-f2995baade0b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:44.160688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde96900-a6e8-4c78-a1f2-688c67c9342d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:44.194273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e2b3a-9efc-4a71-b640-3776a5bada39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:44.194941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e2b3a-9efc-4a71-b640-3776a5bada39 None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:44.195223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e2b3a-9efc-4a71-b640-3776a5bada39 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:44.195500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e2b3a-9efc-4a71-b640-3776a5bada39 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:44.196195 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e2b3a-9efc-4a71-b640-3776a5bada39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:44.197662 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e2b3a-9efc-4a71-b640-3776a5bada39 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:44.219012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 261/519] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:32:44 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:32:44.219756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e2b3a-9efc-4a71-b640-3776a5bada39 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:44.226853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:44.262051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5322dd-9a0f-4300-ace1-f2995baade0b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:44.303901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:44.304092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8c511-ddf4-4840-8393-c4e53a248868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:44.305080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8c511-ddf4-4840-8393-c4e53a248868 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:44.305080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8c511-ddf4-4840-8393-c4e53a248868 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:44.305356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8c511-ddf4-4840-8393-c4e53a248868 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:44.313543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8c511-ddf4-4840-8393-c4e53a248868 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:44.315036 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8c511-ddf4-4840-8393-c4e53a248868 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-94c8c511-ddf4-4840-8393-c4e53a248868 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:44.330927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:44.332095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:44.333593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:44.333593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:44.333593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:44.333593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:44.333593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:44.333593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:44.333593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:44.333593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:44.333593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 08:32:44.333593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:44.333593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 259/520] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:44.334185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8c511-ddf4-4840-8393-c4e53a248868 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:44.350245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e2b3a-9efc-4a71-b640-3776a5bada39 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:44.390123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a73209-e599-48c8-8278-83cd9abc41a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:44.390985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a73209-e599-48c8-8278-83cd9abc41a1 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:44.391271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a73209-e599-48c8-8278-83cd9abc41a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:44.391549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a73209-e599-48c8-8278-83cd9abc41a1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:44.399956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a73209-e599-48c8-8278-83cd9abc41a1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:44.401431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a73209-e599-48c8-8278-83cd9abc41a1 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-b0a73209-e599-48c8-8278-83cd9abc41a1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:44.417881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:44.418970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:44.420040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:44.420040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:44.420040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:44.420040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:44.420040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:44.420040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:44.420040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:44.420040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:44.420040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 08:32:44.420040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:44.420040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 262/521] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:44.420717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a73209-e599-48c8-8278-83cd9abc41a1 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:44.460545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8c511-ddf4-4840-8393-c4e53a248868 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:44.491117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:44.497761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497adecd-3600-4d5c-8536-69f1bbca6a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:44.498606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497adecd-3600-4d5c-8536-69f1bbca6a9a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:44.498959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497adecd-3600-4d5c-8536-69f1bbca6a9a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:44.499233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497adecd-3600-4d5c-8536-69f1bbca6a9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:44.510041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497adecd-3600-4d5c-8536-69f1bbca6a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:44.511510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497adecd-3600-4d5c-8536-69f1bbca6a9a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-497adecd-3600-4d5c-8536-69f1bbca6a9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:44.529116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:44.530138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:44.533313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:44.533313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:44.533313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:44.533313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:44.533313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 260/522] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:44.533313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497adecd-3600-4d5c-8536-69f1bbca6a9a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:44.538329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a73209-e599-48c8-8278-83cd9abc41a1 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:44.574145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:44.581638 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bb442-39b5-4432-835d-9a033fc487d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:44.582334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bb442-39b5-4432-835d-9a033fc487d5 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:44.582656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bb442-39b5-4432-835d-9a033fc487d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:44.582944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bb442-39b5-4432-835d-9a033fc487d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:44.583788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bb442-39b5-4432-835d-9a033fc487d5 None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=8b476d6b82ae464b81fc8498dc0e68c7) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:44.584104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bb442-39b5-4432-835d-9a033fc487d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:44.585568 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bb442-39b5-4432-835d-9a033fc487d5 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:44.592015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 263/523] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:44 2022] GET /identity/v3/role_assignments?user.id=8b476d6b82ae464b81fc8498dc0e68c7&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:44.592840 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bb442-39b5-4432-835d-9a033fc487d5 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:44.659222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497adecd-3600-4d5c-8536-69f1bbca6a9a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:44.702854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d062e-8309-4453-9ef1-78c53a7e0f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:44.703652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d062e-8309-4453-9ef1-78c53a7e0f75 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:44.704160 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d062e-8309-4453-9ef1-78c53a7e0f75 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:44.704467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d062e-8309-4453-9ef1-78c53a7e0f75 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:44.705601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d062e-8309-4453-9ef1-78c53a7e0f75 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:44.705921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d062e-8309-4453-9ef1-78c53a7e0f75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:44.707592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d062e-8309-4453-9ef1-78c53a7e0f75 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:44.713744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bb442-39b5-4432-835d-9a033fc487d5 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:44.733163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 261/524] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Jul 25 08:32:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:44.734309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d062e-8309-4453-9ef1-78c53a7e0f75 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:44.763480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a823b8b-9824-4c0a-b8ff-408dd420da69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:44.764771 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a823b8b-9824-4c0a-b8ff-408dd420da69 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:44.765326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a823b8b-9824-4c0a-b8ff-408dd420da69 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:44.765920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a823b8b-9824-4c0a-b8ff-408dd420da69 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:44.767558 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a823b8b-9824-4c0a-b8ff-408dd420da69 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:44.768056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a823b8b-9824-4c0a-b8ff-408dd420da69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:44.770408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a823b8b-9824-4c0a-b8ff-408dd420da69 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:44.781735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:44.823856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 264/525] 10.222.0.52 () {56 vars in 1139 bytes} [Mon Jul 25 08:32:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:44.825278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a823b8b-9824-4c0a-b8ff-408dd420da69 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:44.851572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d062e-8309-4453-9ef1-78c53a7e0f75 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:44.888425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b07f7b-ed8f-4007-9dad-b12dead1d984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:44.889285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b07f7b-ed8f-4007-9dad-b12dead1d984 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:44.889995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b07f7b-ed8f-4007-9dad-b12dead1d984 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:44.890382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b07f7b-ed8f-4007-9dad-b12dead1d984 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:44.891521 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b07f7b-ed8f-4007-9dad-b12dead1d984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:44.891936 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b07f7b-ed8f-4007-9dad-b12dead1d984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:44.893624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b07f7b-ed8f-4007-9dad-b12dead1d984 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:44.901282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:44.914888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 262/526] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:44.914888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b07f7b-ed8f-4007-9dad-b12dead1d984 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:44.975924 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a823b8b-9824-4c0a-b8ff-408dd420da69 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:45.023501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d984c-696d-4e4d-9dd1-9b622441ef55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:45.024001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d984c-696d-4e4d-9dd1-9b622441ef55 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:45.024293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d984c-696d-4e4d-9dd1-9b622441ef55 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:45.024609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d984c-696d-4e4d-9dd1-9b622441ef55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:45.043651 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:45.048227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b07f7b-ed8f-4007-9dad-b12dead1d984 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:45.092309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3335f2a-6316-456a-a483-b2db32850eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:45.093781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335f2a-6316-456a-a483-b2db32850eea None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:45.094280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335f2a-6316-456a-a483-b2db32850eea None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:45.095066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335f2a-6316-456a-a483-b2db32850eea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:45.111688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d984c-696d-4e4d-9dd1-9b622441ef55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:45.113560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d984c-696d-4e4d-9dd1-9b622441ef55 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:45.186307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:45.187104 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3335f2a-6316-456a-a483-b2db32850eea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:45.190183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3335f2a-6316-456a-a483-b2db32850eea None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:45.250894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 265/527] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:45.250894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d984c-696d-4e4d-9dd1-9b622441ef55 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:45.339046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 263/528] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:45.339991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3335f2a-6316-456a-a483-b2db32850eea None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:45.344900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:45.381493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d984c-696d-4e4d-9dd1-9b622441ef55 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:45.416704 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629055c-95a2-483e-b168-9c0c527a3470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:45.417507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629055c-95a2-483e-b168-9c0c527a3470 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:45.417794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629055c-95a2-483e-b168-9c0c527a3470 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:45.418076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629055c-95a2-483e-b168-9c0c527a3470 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:45.424957 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629055c-95a2-483e-b168-9c0c527a3470 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:45.426934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629055c-95a2-483e-b168-9c0c527a3470 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-8629055c-95a2-483e-b168-9c0c527a3470 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:45.441432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:45.442838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:45.443652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:45.443652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:45.443652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:45.443652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:45.443652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:45.443652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:45.443652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:45.443652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:45.443652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 266/529] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:45.444643 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629055c-95a2-483e-b168-9c0c527a3470 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:45.462821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3335f2a-6316-456a-a483-b2db32850eea None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:45.463827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:45.501968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b8295-6967-45db-b833-9607ff8dec2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:45.502992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b8295-6967-45db-b833-9607ff8dec2e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:45.503518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b8295-6967-45db-b833-9607ff8dec2e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:45.503958 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b8295-6967-45db-b833-9607ff8dec2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:45.515855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b8295-6967-45db-b833-9607ff8dec2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:45.517634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b8295-6967-45db-b833-9607ff8dec2e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-350b8295-6967-45db-b833-9607ff8dec2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:45.537251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:45.538305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:45.539293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:45.539293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:45.539293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:45.539293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:45.541216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 264/530] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:45.542211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b8295-6967-45db-b833-9607ff8dec2e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:45.554080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629055c-95a2-483e-b168-9c0c527a3470 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:45.588745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364d91d-583e-41d1-910d-d757b7fa7e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:45.589605 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364d91d-583e-41d1-910d-d757b7fa7e94 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:45.589785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364d91d-583e-41d1-910d-d757b7fa7e94 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:45.590028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364d91d-583e-41d1-910d-d757b7fa7e94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:45.600596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:45.601823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364d91d-583e-41d1-910d-d757b7fa7e94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:45.603029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364d91d-583e-41d1-910d-d757b7fa7e94 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-5364d91d-583e-41d1-910d-d757b7fa7e94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:45.621764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:45.622633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:45.623460 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:45.623460 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:45.623460 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:45.623460 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:45.627944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 267/531] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:45.628873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364d91d-583e-41d1-910d-d757b7fa7e94 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:45.656696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b8295-6967-45db-b833-9607ff8dec2e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:45.694804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:45.697874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3614-4cdd-4e41-b078-2b0b46f15bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:45.699336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3614-4cdd-4e41-b078-2b0b46f15bf7 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:45.699871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3614-4cdd-4e41-b078-2b0b46f15bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:45.701691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3614-4cdd-4e41-b078-2b0b46f15bf7 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:45.703112 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a3614-4cdd-4e41-b078-2b0b46f15bf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:45.703645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a3614-4cdd-4e41-b078-2b0b46f15bf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:45.706776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a3614-4cdd-4e41-b078-2b0b46f15bf7 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:45.730414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 265/532] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:45.730414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3614-4cdd-4e41-b078-2b0b46f15bf7 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:45.748482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364d91d-583e-41d1-910d-d757b7fa7e94 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:45.797455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee8586-db52-4f64-beda-79c4159a51c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:45.798156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee8586-db52-4f64-beda-79c4159a51c3 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:45.798450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee8586-db52-4f64-beda-79c4159a51c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:45.798730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee8586-db52-4f64-beda-79c4159a51c3 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:45.799465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee8586-db52-4f64-beda-79c4159a51c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:45.799774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee8586-db52-4f64-beda-79c4159a51c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:45.801972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee8586-db52-4f64-beda-79c4159a51c3 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:45.814057 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 268/533] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:45.814938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee8586-db52-4f64-beda-79c4159a51c3 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:45.858209 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3614-4cdd-4e41-b078-2b0b46f15bf7 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:45.876689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:45.907298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5059f38-cfa2-44bf-9f78-f6fd0f401027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:45.908238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5059f38-cfa2-44bf-9f78-f6fd0f401027 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:45.908701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5059f38-cfa2-44bf-9f78-f6fd0f401027 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:45.909175 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5059f38-cfa2-44bf-9f78-f6fd0f401027 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:45.910044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5059f38-cfa2-44bf-9f78-f6fd0f401027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:45.911454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5059f38-cfa2-44bf-9f78-f6fd0f401027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:45.913055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5059f38-cfa2-44bf-9f78-f6fd0f401027 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:45.927969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 266/534] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:45.929131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5059f38-cfa2-44bf-9f78-f6fd0f401027 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:45.946676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee8586-db52-4f64-beda-79c4159a51c3 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:45.978626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:45.997596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77580eef-41bf-451e-bb63-154461c2b0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:45.998258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77580eef-41bf-451e-bb63-154461c2b0b5 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:45.998545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77580eef-41bf-451e-bb63-154461c2b0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:45.998828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77580eef-41bf-451e-bb63-154461c2b0b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:45.999669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77580eef-41bf-451e-bb63-154461c2b0b5 None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=925bf2da4df74e86aa84ec62a267d088) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:45.999984 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77580eef-41bf-451e-bb63-154461c2b0b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:46.001449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77580eef-41bf-451e-bb63-154461c2b0b5 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:46.016611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 269/535] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:45 2022] GET /identity/v3/role_assignments?user.id=925bf2da4df74e86aa84ec62a267d088&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:46.016611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77580eef-41bf-451e-bb63-154461c2b0b5 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:46.103074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5059f38-cfa2-44bf-9f78-f6fd0f401027 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:46.180430 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a23354-b56d-463a-b329-52458d8b39f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:46.181862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a23354-b56d-463a-b329-52458d8b39f9 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:46.183943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a23354-b56d-463a-b329-52458d8b39f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:46.184439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a23354-b56d-463a-b329-52458d8b39f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:46.196957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a23354-b56d-463a-b329-52458d8b39f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:46.201887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a23354-b56d-463a-b329-52458d8b39f9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:46.218574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77580eef-41bf-451e-bb63-154461c2b0b5 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-52a23354-b56d-463a-b329-52458d8b39f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:46.225433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:46.226282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:46.227149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:46.227149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:46.227149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:46.227149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:46.227149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:46.227149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:46.227149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:46.227149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:46.229639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 267/536] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:46.235418 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a23354-b56d-463a-b329-52458d8b39f9 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:46.259212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcb9c5-7545-4f23-8e52-2816e411eaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:46.260355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb9c5-7545-4f23-8e52-2816e411eaa2 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:46.260355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb9c5-7545-4f23-8e52-2816e411eaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:46.260652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb9c5-7545-4f23-8e52-2816e411eaa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:46.267928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb9c5-7545-4f23-8e52-2816e411eaa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:46.270966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb9c5-7545-4f23-8e52-2816e411eaa2 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:46.271938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-89bcb9c5-7545-4f23-8e52-2816e411eaa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:46.287096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:46.287988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:46.295356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:46.295356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:46.295356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:46.295356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:46.295356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:46.295356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:46.295356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:46.295356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:46.295356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 270/537] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:46.297173 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcb9c5-7545-4f23-8e52-2816e411eaa2 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:46.372936 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:46.373616 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a23354-b56d-463a-b329-52458d8b39f9 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:46.427355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a42b9a-39f4-4a30-b061-6f098d677fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:46.428032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a42b9a-39f4-4a30-b061-6f098d677fca None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:46.428141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcb9c5-7545-4f23-8e52-2816e411eaa2 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:46.428369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a42b9a-39f4-4a30-b061-6f098d677fca None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:46.428672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a42b9a-39f4-4a30-b061-6f098d677fca None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:46.429735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a42b9a-39f4-4a30-b061-6f098d677fca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:46.430050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a42b9a-39f4-4a30-b061-6f098d677fca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:46.432493 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a42b9a-39f4-4a30-b061-6f098d677fca None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:46.452660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 268/538] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:46.454060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6689c51-dd87-4aea-a95e-a82c7b829c79 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:46.454348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6689c51-dd87-4aea-a95e-a82c7b829c79 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:46.454623 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6689c51-dd87-4aea-a95e-a82c7b829c79 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:46.456218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 269/539] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:46.461031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6689c51-dd87-4aea-a95e-a82c7b829c79 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:46.472028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19914178-55e9-4c97-bb58-da9af6b92446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:46.473222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19914178-55e9-4c97-bb58-da9af6b92446 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:46.473512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19914178-55e9-4c97-bb58-da9af6b92446 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:46.473797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19914178-55e9-4c97-bb58-da9af6b92446 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:46.475184 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19914178-55e9-4c97-bb58-da9af6b92446 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:46.475638 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19914178-55e9-4c97-bb58-da9af6b92446 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:46.478170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19914178-55e9-4c97-bb58-da9af6b92446 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:46.504703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 271/540] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:46.506122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527dbc1-0105-4d89-8fd7-90469e689d99 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:46.506873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527dbc1-0105-4d89-8fd7-90469e689d99 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:46.507160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527dbc1-0105-4d89-8fd7-90469e689d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:46.532503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7527dbc1-0105-4d89-8fd7-90469e689d99 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:46.534659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7527dbc1-0105-4d89-8fd7-90469e689d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:46.560196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:46.576094 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6689c51-dd87-4aea-a95e-a82c7b829c79 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:46.616829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 272/541] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:46.617908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527dbc1-0105-4d89-8fd7-90469e689d99 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:46.620443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205f03b-291c-4883-b437-95b0acf29bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:46.621171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f03b-291c-4883-b437-95b0acf29bb6 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:46.621458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f03b-291c-4883-b437-95b0acf29bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:46.621735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f03b-291c-4883-b437-95b0acf29bb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:46.622577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205f03b-291c-4883-b437-95b0acf29bb6 None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=84871b711bcb47efb039ce40766cf3db) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:46.622900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205f03b-291c-4883-b437-95b0acf29bb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:46.624319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205f03b-291c-4883-b437-95b0acf29bb6 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:46.636243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 270/542] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:46 2022] GET /identity/v3/role_assignments?user.id=84871b711bcb47efb039ce40766cf3db&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:46.638241 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f440cfd-3ab7-4c57-8eb4-ae85f6125783 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:46.638531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f440cfd-3ab7-4c57-8eb4-ae85f6125783 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:46.638807 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f440cfd-3ab7-4c57-8eb4-ae85f6125783 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:46.640367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 271/543] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:46.645855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f440cfd-3ab7-4c57-8eb4-ae85f6125783 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:46.681697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:46.755757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527dbc1-0105-4d89-8fd7-90469e689d99 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:46.762019 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f440cfd-3ab7-4c57-8eb4-ae85f6125783 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:46.799531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f72f89-b2fe-4ffc-b4ad-90a25d014c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:46.800656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f72f89-b2fe-4ffc-b4ad-90a25d014c0b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:46.801676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f72f89-b2fe-4ffc-b4ad-90a25d014c0b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:46.802192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f72f89-b2fe-4ffc-b4ad-90a25d014c0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:46.803264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f72f89-b2fe-4ffc-b4ad-90a25d014c0b None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=62688bc3d8a449c5bc9e0d64cd2462b2) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:46.803782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f72f89-b2fe-4ffc-b4ad-90a25d014c0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:46.805477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f72f89-b2fe-4ffc-b4ad-90a25d014c0b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:46.811060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85353f2d-71e1-451f-9b2a-2122f91eb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:46.813134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85353f2d-71e1-451f-9b2a-2122f91eb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:46.815015 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85353f2d-71e1-451f-9b2a-2122f91eb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:46.815015 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85353f2d-71e1-451f-9b2a-2122f91eb2b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:46.817562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 273/544] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:46 2022] GET /identity/v3/role_assignments?user.id=62688bc3d8a449c5bc9e0d64cd2462b2&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:46.819324 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9a6c6-6ca3-47de-8fea-47e70294e0f4 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:46.819618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9a6c6-6ca3-47de-8fea-47e70294e0f4 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:46.819899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9a6c6-6ca3-47de-8fea-47e70294e0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:46.832328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85353f2d-71e1-451f-9b2a-2122f91eb2b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:46.839866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-5db9a6c6-6ca3-47de-8fea-47e70294e0f4 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:46.841144 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-5db9a6c6-6ca3-47de-8fea-47e70294e0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-85353f2d-71e1-451f-9b2a-2122f91eb2b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:46.842944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:46.843810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:46.844673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:46.844673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:46.844673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:46.844673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:46.844673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:46.844673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:46.844673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:46.844673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:46.844673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:46.844673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:46.844673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:46.844673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:46.846873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 272/545] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:46.852953 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85353f2d-71e1-451f-9b2a-2122f91eb2b7 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:46.875667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:46.924593 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 274/546] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:46.944800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9a6c6-6ca3-47de-8fea-47e70294e0f4 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:47.003882 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:47.040186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85353f2d-71e1-451f-9b2a-2122f91eb2b7 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:47.084603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c76063-2dfe-4351-bb07-13228d6278e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:47.085163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c76063-2dfe-4351-bb07-13228d6278e0 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:47.085449 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c76063-2dfe-4351-bb07-13228d6278e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:47.085724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c76063-2dfe-4351-bb07-13228d6278e0 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:47.086492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c76063-2dfe-4351-bb07-13228d6278e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:47.086821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c76063-2dfe-4351-bb07-13228d6278e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:47.088180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c76063-2dfe-4351-bb07-13228d6278e0 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:47.099128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9a6c6-6ca3-47de-8fea-47e70294e0f4 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:47.105398 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 273/547] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:47.110462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c76063-2dfe-4351-bb07-13228d6278e0 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:47.140242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a2ea0-fbb3-45df-8bbf-8a277623c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:47.141482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a2ea0-fbb3-45df-8bbf-8a277623c22f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:47.141962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a2ea0-fbb3-45df-8bbf-8a277623c22f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:47.142450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a2ea0-fbb3-45df-8bbf-8a277623c22f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:47.154253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a2ea0-fbb3-45df-8bbf-8a277623c22f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-790a2ea0-fbb3-45df-8bbf-8a277623c22f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:47.164066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:47.164979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:47.165752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:47.165752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:47.165752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:47.165752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:47.165752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:47.165752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:47.165752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:47.165752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:47.165752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:47.165752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:47.165752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:47.165752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:47.172825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 275/548] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:47.174666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a2ea0-fbb3-45df-8bbf-8a277623c22f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:47.244532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:47.279096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:47.289604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c76063-2dfe-4351-bb07-13228d6278e0 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:47.290274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a2ea0-fbb3-45df-8bbf-8a277623c22f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:47.326855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba0959-4e89-4558-8ac4-195bbac2001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:47.327839 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba0959-4e89-4558-8ac4-195bbac2001f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:47.328324 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba0959-4e89-4558-8ac4-195bbac2001f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:47.328928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba0959-4e89-4558-8ac4-195bbac2001f None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:47.329885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba0959-4e89-4558-8ac4-195bbac2001f None admin] RBAC: Adding query filter params (name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:47.330388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba0959-4e89-4558-8ac4-195bbac2001f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:47.332137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba0959-4e89-4558-8ac4-195bbac2001f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:47.332326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0722082-998e-45b9-88fc-80b610d606aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:47.333289 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0722082-998e-45b9-88fc-80b610d606aa None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:47.333639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0722082-998e-45b9-88fc-80b610d606aa None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:47.333981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0722082-998e-45b9-88fc-80b610d606aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:47.345492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0722082-998e-45b9-88fc-80b610d606aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:47.345939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 276/549] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:47.347123 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0722082-998e-45b9-88fc-80b610d606aa None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:47.348835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba0959-4e89-4558-8ac4-195bbac2001f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-e0722082-998e-45b9-88fc-80b610d606aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:47.367618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:47.369099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:47.370549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:47.370549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:47.370549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:47.370549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:47.370549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 274/550] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:47.381901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0722082-998e-45b9-88fc-80b610d606aa None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:47.515980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0722082-998e-45b9-88fc-80b610d606aa None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:47.551354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:47.554949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396ac00-a024-4916-bcc5-f0de3c05ca71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:47.555616 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ac00-a024-4916-bcc5-f0de3c05ca71 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:47.555901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ac00-a024-4916-bcc5-f0de3c05ca71 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:47.556282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ac00-a024-4916-bcc5-f0de3c05ca71 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:47.557054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396ac00-a024-4916-bcc5-f0de3c05ca71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:47.557385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396ac00-a024-4916-bcc5-f0de3c05ca71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:47.559217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396ac00-a024-4916-bcc5-f0de3c05ca71 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:47.572180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 275/551] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:47.577850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396ac00-a024-4916-bcc5-f0de3c05ca71 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:47.642553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba0959-4e89-4558-8ac4-195bbac2001f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:47.684098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396ac00-a024-4916-bcc5-f0de3c05ca71 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:47.720750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eab40c-bfe2-4d21-b770-71d921a23c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:47.721547 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eab40c-bfe2-4d21-b770-71d921a23c90 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:47.721831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eab40c-bfe2-4d21-b770-71d921a23c90 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:47.722101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eab40c-bfe2-4d21-b770-71d921a23c90 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:47.730047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eab40c-bfe2-4d21-b770-71d921a23c90 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:47.731473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eab40c-bfe2-4d21-b770-71d921a23c90 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:47.737261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547f036-fa0c-4e7b-bbad-2a130d3d9f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:47.738074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547f036-fa0c-4e7b-bbad-2a130d3d9f9f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:47.738362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547f036-fa0c-4e7b-bbad-2a130d3d9f9f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:47.738674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547f036-fa0c-4e7b-bbad-2a130d3d9f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-40eab40c-bfe2-4d21-b770-71d921a23c90 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:47.746121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:47.747079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:47.748469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:47.748469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:47.748469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:47.748469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:47.748469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:47.748469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:47.748469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:47.748469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:47.748469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 08:32:47.748469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:47.748469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 276/552] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:32:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:47.753214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eab40c-bfe2-4d21-b770-71d921a23c90 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:47.768059 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547f036-fa0c-4e7b-bbad-2a130d3d9f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:47.769580 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547f036-fa0c-4e7b-bbad-2a130d3d9f9f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-4547f036-fa0c-4e7b-bbad-2a130d3d9f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:47.813378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:47.815504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:47.818100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:47.818100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:47.818100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:47.818100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:47.818100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 277/553] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 467 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:47.824132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547f036-fa0c-4e7b-bbad-2a130d3d9f9f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:47.843384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:47.857122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:47.859513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eab40c-bfe2-4d21-b770-71d921a23c90 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:47.930169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee329cb9-27e0-4dc4-9673-8357300c510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:47.931448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329cb9-27e0-4dc4-9673-8357300c510f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:47.931734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329cb9-27e0-4dc4-9673-8357300c510f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:47.932010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329cb9-27e0-4dc4-9673-8357300c510f None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:47.932887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee329cb9-27e0-4dc4-9673-8357300c510f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:47.933196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee329cb9-27e0-4dc4-9673-8357300c510f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:47.934630 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee329cb9-27e0-4dc4-9673-8357300c510f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:47.966425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 277/554] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Jul 25 08:32:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:47.973599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee329cb9-27e0-4dc4-9673-8357300c510f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:47.992041 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547f036-fa0c-4e7b-bbad-2a130d3d9f9f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:48.032827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d95456-7808-4407-b899-a3c88432091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:48.033669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d95456-7808-4407-b899-a3c88432091b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:48.033962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d95456-7808-4407-b899-a3c88432091b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:48.034236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d95456-7808-4407-b899-a3c88432091b None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:48.034967 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d95456-7808-4407-b899-a3c88432091b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:48.035270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d95456-7808-4407-b899-a3c88432091b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:48.036710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d95456-7808-4407-b899-a3c88432091b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:48.058076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 278/555] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:48.063789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d95456-7808-4407-b899-a3c88432091b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:48.111994 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee329cb9-27e0-4dc4-9673-8357300c510f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:48.125871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:48.156548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547cee7-fde5-4fff-b45d-2bb8ab484fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:48.157233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547cee7-fde5-4fff-b45d-2bb8ab484fa7 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:48.157517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547cee7-fde5-4fff-b45d-2bb8ab484fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:48.158142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547cee7-fde5-4fff-b45d-2bb8ab484fa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:48.199392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:48.261214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d95456-7808-4407-b899-a3c88432091b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:48.269393 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547cee7-fde5-4fff-b45d-2bb8ab484fa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:48.271427 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547cee7-fde5-4fff-b45d-2bb8ab484fa7 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:48.316225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f6d9b-9119-40f6-aaf2-33e8b87875af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:48.317556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f6d9b-9119-40f6-aaf2-33e8b87875af None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:48.317847 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f6d9b-9119-40f6-aaf2-33e8b87875af None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:48.318125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f6d9b-9119-40f6-aaf2-33e8b87875af None admin] PATH_INFO: `/v3/users/nova` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:48.327471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f6d9b-9119-40f6-aaf2-33e8b87875af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:48.329676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f6d9b-9119-40f6-aaf2-33e8b87875af None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-d78f6d9b-9119-40f6-aaf2-33e8b87875af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:48.349689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:48.354883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 279/556] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Jul 25 08:32:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9adfd-1ddd-4c01-bf91-97d690a22e68 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:48.356011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9adfd-1ddd-4c01-bf91-97d690a22e68 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:48.356870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9adfd-1ddd-4c01-bf91-97d690a22e68 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:48.356870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 280/557] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:48.361917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9adfd-1ddd-4c01-bf91-97d690a22e68 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:48.406139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 278/558] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:48.407941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91811ef-4a9d-4101-931a-752aa34e6c4f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:48.408238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91811ef-4a9d-4101-931a-752aa34e6c4f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:48.408523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91811ef-4a9d-4101-931a-752aa34e6c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:48.433615 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-f91811ef-4a9d-4101-931a-752aa34e6c4f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:48.435420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f91811ef-4a9d-4101-931a-752aa34e6c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:48.438053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:48.474414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9adfd-1ddd-4c01-bf91-97d690a22e68 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:48.505587 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:48.519079 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476cda9-2954-47b2-9b65-ff0ade20e1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:48.520766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476cda9-2954-47b2-9b65-ff0ade20e1cd None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:48.521267 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476cda9-2954-47b2-9b65-ff0ade20e1cd None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:48.522012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476cda9-2954-47b2-9b65-ff0ade20e1cd None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:48.523770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476cda9-2954-47b2-9b65-ff0ade20e1cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:48.524314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476cda9-2954-47b2-9b65-ff0ade20e1cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:48.525976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476cda9-2954-47b2-9b65-ff0ade20e1cd None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:48.532668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 279/559] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:48.533704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91811ef-4a9d-4101-931a-752aa34e6c4f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:48.547527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 281/560] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Jul 25 08:32:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:48.561916 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476cda9-2954-47b2-9b65-ff0ade20e1cd None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:48.654264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91811ef-4a9d-4101-931a-752aa34e6c4f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:48.667015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476cda9-2954-47b2-9b65-ff0ade20e1cd None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:48.690336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6105f2d-9caa-49d5-b7ea-4c8e42e88661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:48.691135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6105f2d-9caa-49d5-b7ea-4c8e42e88661 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:48.691415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6105f2d-9caa-49d5-b7ea-4c8e42e88661 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:48.691730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6105f2d-9caa-49d5-b7ea-4c8e42e88661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:48.698770 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:48.703648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6105f2d-9caa-49d5-b7ea-4c8e42e88661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:48.705201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2c53a-6a69-4ae1-a9aa-31bd7c6ea2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:48.705334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6105f2d-9caa-49d5-b7ea-4c8e42e88661 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:48.706351 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2c53a-6a69-4ae1-a9aa-31bd7c6ea2d8 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:48.706845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2c53a-6a69-4ae1-a9aa-31bd7c6ea2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:48.707385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2c53a-6a69-4ae1-a9aa-31bd7c6ea2d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:48.718622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2c53a-6a69-4ae1-a9aa-31bd7c6ea2d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:48.720325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2c53a-6a69-4ae1-a9aa-31bd7c6ea2d8 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-f6105f2d-9caa-49d5-b7ea-4c8e42e88661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:48.725085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:48.725960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:48.726834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:48.726834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:48.726834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:48.726834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:48.726834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 280/561] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:48.727566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6105f2d-9caa-49d5-b7ea-4c8e42e88661 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-23c2c53a-6a69-4ae1-a9aa-31bd7c6ea2d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:48.742507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:48.743505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:48.744360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:48.744360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:48.744360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:48.744360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:48.750937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 282/562] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:48.752068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2c53a-6a69-4ae1-a9aa-31bd7c6ea2d8 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:48.790503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:48.837554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6105f2d-9caa-49d5-b7ea-4c8e42e88661 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:48.860544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2c53a-6a69-4ae1-a9aa-31bd7c6ea2d8 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:48.874129 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d85bda-38df-4b5e-bc5b-fff87345ef85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:48.874930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d85bda-38df-4b5e-bc5b-fff87345ef85 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:48.875210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d85bda-38df-4b5e-bc5b-fff87345ef85 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:48.875486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d85bda-38df-4b5e-bc5b-fff87345ef85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:48.887681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d85bda-38df-4b5e-bc5b-fff87345ef85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-04d85bda-38df-4b5e-bc5b-fff87345ef85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:48.896734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:48.897686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:48.898515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:48.898515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:48.898515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:48.898515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:48.898515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:48.898515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:48.898515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:48.898515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:48.898515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:48.898515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:48.898515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:48.898515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:48.900657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 281/563] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:48.901689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d85bda-38df-4b5e-bc5b-fff87345ef85 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:48.910379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a257c-f74f-498c-8504-8944a6848646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:48.911393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a257c-f74f-498c-8504-8944a6848646 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:48.911883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a257c-f74f-498c-8504-8944a6848646 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:48.912378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a257c-f74f-498c-8504-8944a6848646 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:48.913463 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a257c-f74f-498c-8504-8944a6848646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:48.914034 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a257c-f74f-498c-8504-8944a6848646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:48.915859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a257c-f74f-498c-8504-8944a6848646 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:48.929966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 283/564] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:48.931141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a257c-f74f-498c-8504-8944a6848646 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:48.956688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:49.011815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d85bda-38df-4b5e-bc5b-fff87345ef85 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:49.038357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:49.045668 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a257c-f74f-498c-8504-8944a6848646 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:49.048373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27275374-f81f-4745-8309-0cf5666b4304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:49.049074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27275374-f81f-4745-8309-0cf5666b4304 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.049365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27275374-f81f-4745-8309-0cf5666b4304 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.049643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27275374-f81f-4745-8309-0cf5666b4304 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.050375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27275374-f81f-4745-8309-0cf5666b4304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:49.050684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27275374-f81f-4745-8309-0cf5666b4304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:49.052607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27275374-f81f-4745-8309-0cf5666b4304 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:49.069048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 282/565] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:49.070822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27275374-f81f-4745-8309-0cf5666b4304 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:49.080083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecd4bc-40c8-4cff-90a1-7cc76aec5bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:49.080781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecd4bc-40c8-4cff-90a1-7cc76aec5bac None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.081068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecd4bc-40c8-4cff-90a1-7cc76aec5bac None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.081343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecd4bc-40c8-4cff-90a1-7cc76aec5bac None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.082090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecd4bc-40c8-4cff-90a1-7cc76aec5bac None admin] RBAC: Adding query filter params (name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:49.082431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecd4bc-40c8-4cff-90a1-7cc76aec5bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:49.083916 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecd4bc-40c8-4cff-90a1-7cc76aec5bac None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:49.093669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 284/566] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:49.094966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075a759-4ec2-4126-a99f-71ee3c643a9c None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.095255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075a759-4ec2-4126-a99f-71ee3c643a9c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.095535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075a759-4ec2-4126-a99f-71ee3c643a9c None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.100863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 285/567] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:49.101623 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075a759-4ec2-4126-a99f-71ee3c643a9c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:49.175589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27275374-f81f-4745-8309-0cf5666b4304 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:49.186605 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:49.210599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c714f-59d6-4703-baf6-acd650738e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:49.212009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c714f-59d6-4703-baf6-acd650738e48 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.212266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c714f-59d6-4703-baf6-acd650738e48 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.212488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c714f-59d6-4703-baf6-acd650738e48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.219491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c714f-59d6-4703-baf6-acd650738e48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:49.220968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c714f-59d6-4703-baf6-acd650738e48 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:49.229818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075a759-4ec2-4126-a99f-71ee3c643a9c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-ed6c714f-59d6-4703-baf6-acd650738e48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:49.234690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:49.235546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:49.236464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:49.236464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:49.236464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:49.236464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:49.236464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:49.236464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:49.236464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:49.236464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:49.236464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 283/568] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:49.237050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c714f-59d6-4703-baf6-acd650738e48 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:49.266788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424f7f6-78d5-4825-a8de-fa94b420a586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:49.267640 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424f7f6-78d5-4825-a8de-fa94b420a586 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.267931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424f7f6-78d5-4825-a8de-fa94b420a586 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.268261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424f7f6-78d5-4825-a8de-fa94b420a586 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.274942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424f7f6-78d5-4825-a8de-fa94b420a586 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:49.276377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424f7f6-78d5-4825-a8de-fa94b420a586 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-6424f7f6-78d5-4825-a8de-fa94b420a586 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:49.290748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:49.291810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:49.292933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:49.292933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:49.292933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:49.292933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:49.292933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:49.292933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:49.292933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:49.292933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:49.296494 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 286/569] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:49.297727 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831b951-9e0e-4468-89b7-4a15b98d6f38 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.298019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831b951-9e0e-4468-89b7-4a15b98d6f38 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.298295 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831b951-9e0e-4468-89b7-4a15b98d6f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.316228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-f831b951-9e0e-4468-89b7-4a15b98d6f38 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:49.317981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f831b951-9e0e-4468-89b7-4a15b98d6f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:49.330970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:49.343705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c714f-59d6-4703-baf6-acd650738e48 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:49.376177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f1340-188c-4ce6-902f-01a54e616439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:49.377037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f1340-188c-4ce6-902f-01a54e616439 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.377333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f1340-188c-4ce6-902f-01a54e616439 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.377618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f1340-188c-4ce6-902f-01a54e616439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.385079 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 287/570] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:49.386448 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb60d36-ab09-4733-9b65-6deb6e43c72d None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.386744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb60d36-ab09-4733-9b65-6deb6e43c72d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.387020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb60d36-ab09-4733-9b65-6deb6e43c72d None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.388563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 288/571] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:49.389150 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f1340-188c-4ce6-902f-01a54e616439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:49.389284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb60d36-ab09-4733-9b65-6deb6e43c72d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:49.390976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f1340-188c-4ce6-902f-01a54e616439 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-d17f1340-188c-4ce6-902f-01a54e616439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:49.410570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:49.411656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:49.412806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:49.412806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:49.412806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:49.412806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:49.412806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 284/572] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:49.412806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f1340-188c-4ce6-902f-01a54e616439 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:49.436619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:49.500639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb60d36-ab09-4733-9b65-6deb6e43c72d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:49.513379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f1340-188c-4ce6-902f-01a54e616439 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:49.534913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea521115-5a9f-4b13-935d-5423a8c7154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:49.535599 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea521115-5a9f-4b13-935d-5423a8c7154c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.535882 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea521115-5a9f-4b13-935d-5423a8c7154c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.536159 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea521115-5a9f-4b13-935d-5423a8c7154c None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.536978 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea521115-5a9f-4b13-935d-5423a8c7154c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:49.537289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea521115-5a9f-4b13-935d-5423a8c7154c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:49.538722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea521115-5a9f-4b13-935d-5423a8c7154c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:49.546578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47070bf9-46fb-4074-a2c2-733e36a84fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:49.547226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47070bf9-46fb-4074-a2c2-733e36a84fb2 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.547508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47070bf9-46fb-4074-a2c2-733e36a84fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.547801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47070bf9-46fb-4074-a2c2-733e36a84fb2 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.548599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47070bf9-46fb-4074-a2c2-733e36a84fb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:49.548914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47070bf9-46fb-4074-a2c2-733e36a84fb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:49.550020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 289/573] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:49.550316 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47070bf9-46fb-4074-a2c2-733e36a84fb2 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:49.551226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1de1b7-3624-4a8a-964c-e679ae7d9270 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.551514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1de1b7-3624-4a8a-964c-e679ae7d9270 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.551796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1de1b7-3624-4a8a-964c-e679ae7d9270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.565629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 285/574] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:49.566279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47070bf9-46fb-4074-a2c2-733e36a84fb2 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:49.569241 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-1f1de1b7-3624-4a8a-964c-e679ae7d9270 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:49.570431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1f1de1b7-3624-4a8a-964c-e679ae7d9270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:49.572464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:49.637375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 290/575] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:49.638198 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1de1b7-3624-4a8a-964c-e679ae7d9270 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:49.659113 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:49.667749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47070bf9-46fb-4074-a2c2-733e36a84fb2 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:49.702306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6c2d5-a7d8-47da-b1aa-b40b216e3740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:49.703043 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6c2d5-a7d8-47da-b1aa-b40b216e3740 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.703332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6c2d5-a7d8-47da-b1aa-b40b216e3740 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.703615 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6c2d5-a7d8-47da-b1aa-b40b216e3740 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.713680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6c2d5-a7d8-47da-b1aa-b40b216e3740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-54b6c2d5-a7d8-47da-b1aa-b40b216e3740 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:49.721446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:49.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 286/576] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:49.728253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6c2d5-a7d8-47da-b1aa-b40b216e3740 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:49.738255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1de1b7-3624-4a8a-964c-e679ae7d9270 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:49.773406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92c681-d44b-4473-85d7-1bf37f53b744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:49.774074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92c681-d44b-4473-85d7-1bf37f53b744 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.774356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92c681-d44b-4473-85d7-1bf37f53b744 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.774635 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92c681-d44b-4473-85d7-1bf37f53b744 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.775350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92c681-d44b-4473-85d7-1bf37f53b744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:49.775652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92c681-d44b-4473-85d7-1bf37f53b744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:49.777050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92c681-d44b-4473-85d7-1bf37f53b744 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:49.783538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:49.789097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 291/577] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:49.789914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92c681-d44b-4473-85d7-1bf37f53b744 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:49.829200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6c2d5-a7d8-47da-b1aa-b40b216e3740 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:49.861298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c8e0b-e14b-4d54-b50c-395a2dd34c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:49.861950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c8e0b-e14b-4d54-b50c-395a2dd34c3b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.862232 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c8e0b-e14b-4d54-b50c-395a2dd34c3b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.862506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c8e0b-e14b-4d54-b50c-395a2dd34c3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.863836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c8e0b-e14b-4d54-b50c-395a2dd34c3b None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=7d6be43abd85417c9fb7bbcb36a1f77d) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:49.864156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c8e0b-e14b-4d54-b50c-395a2dd34c3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:49.865600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c8e0b-e14b-4d54-b50c-395a2dd34c3b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:49.871046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 287/578] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:49 2022] GET /identity/v3/role_assignments?user.id=7d6be43abd85417c9fb7bbcb36a1f77d&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:49.871888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c8e0b-e14b-4d54-b50c-395a2dd34c3b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:49.889754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92c681-d44b-4473-85d7-1bf37f53b744 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:49.896496 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:49.928872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2718a4e-4336-4382-a57f-1f085d228cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:49.929924 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2718a4e-4336-4382-a57f-1f085d228cf9 None admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:49.930275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2718a4e-4336-4382-a57f-1f085d228cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:49.930628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2718a4e-4336-4382-a57f-1f085d228cf9 None admin] PATH_INFO: `/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/8b476d6b82ae464b81fc8498dc0e68c7/roles/45565d93c8c142e086e6b7ce62ff84e4` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:49.948200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2718a4e-4336-4382-a57f-1f085d228cf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ee27c4da77c47428e02fd65e890d256, user_id=8b476d6b82ae464b81fc8498dc0e68c7, role_id=45565d93c8c142e086e6b7ce62ff84e4)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:49.949825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2718a4e-4336-4382-a57f-1f085d228cf9 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:49.965584 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 292/579] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:32:49 2022] PUT /identity/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/8b476d6b82ae464b81fc8498dc0e68c7/roles/45565d93c8c142e086e6b7ce62ff84e4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:49.966831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2718a4e-4336-4382-a57f-1f085d228cf9 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:50.022436 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c8e0b-e14b-4d54-b50c-395a2dd34c3b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:50.054395 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:50.075156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfde1c-1e9f-4e6c-bdb0-4421bd46e42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:50.076075 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfde1c-1e9f-4e6c-bdb0-4421bd46e42d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:50.076444 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfde1c-1e9f-4e6c-bdb0-4421bd46e42d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:50.077470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfde1c-1e9f-4e6c-bdb0-4421bd46e42d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:50.092937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfde1c-1e9f-4e6c-bdb0-4421bd46e42d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-34dfde1c-1e9f-4e6c-bdb0-4421bd46e42d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:50.104486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:50.105728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:50.106781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:50.106781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:50.106781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:50.106781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:50.106781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:50.106781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:50.106781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:50.106781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:50.106781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:50.106781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:50.106781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:50.106781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:50.107788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 288/580] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:50.109087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfde1c-1e9f-4e6c-bdb0-4421bd46e42d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:50.121327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2718a4e-4336-4382-a57f-1f085d228cf9 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:50.162494 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9eccda-d9df-435c-a5ba-39bf8330a68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:50.162705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eccda-d9df-435c-a5ba-39bf8330a68c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:50.162965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eccda-d9df-435c-a5ba-39bf8330a68c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:50.163239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eccda-d9df-435c-a5ba-39bf8330a68c None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:50.163990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9eccda-d9df-435c-a5ba-39bf8330a68c None admin] RBAC: Adding query filter params (name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:50.164296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9eccda-d9df-435c-a5ba-39bf8330a68c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:50.165717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9eccda-d9df-435c-a5ba-39bf8330a68c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:50.176603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 293/581] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:50.176603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9eccda-d9df-435c-a5ba-39bf8330a68c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:50.201489 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:50.241328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfde1c-1e9f-4e6c-bdb0-4421bd46e42d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:50.277852 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344e0de-4910-40d7-a006-25bcb5b51865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:50.279212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344e0de-4910-40d7-a006-25bcb5b51865 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:50.279680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344e0de-4910-40d7-a006-25bcb5b51865 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:50.280230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344e0de-4910-40d7-a006-25bcb5b51865 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:50.287730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9eccda-d9df-435c-a5ba-39bf8330a68c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:50.289411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344e0de-4910-40d7-a006-25bcb5b51865 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:50.291230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344e0de-4910-40d7-a006-25bcb5b51865 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:50.299429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-2344e0de-4910-40d7-a006-25bcb5b51865 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:50.308273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:50.309474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:50.310421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:50.310421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:50.310421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:50.310421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:50.310421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:50.310421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:50.310421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:50.310421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:50.310421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 08:32:50.310421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:50.313830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 289/582] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:50.314905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344e0de-4910-40d7-a006-25bcb5b51865 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:50.325349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5784d55-1a45-4498-86ff-f0561f44a9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:50.326205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5784d55-1a45-4498-86ff-f0561f44a9e5 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:50.326495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5784d55-1a45-4498-86ff-f0561f44a9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:50.326769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5784d55-1a45-4498-86ff-f0561f44a9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:50.327579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5784d55-1a45-4498-86ff-f0561f44a9e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:50.327890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5784d55-1a45-4498-86ff-f0561f44a9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:50.329431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5784d55-1a45-4498-86ff-f0561f44a9e5 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:50.344859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 294/583] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:50.344859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5784d55-1a45-4498-86ff-f0561f44a9e5 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:50.442953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5784d55-1a45-4498-86ff-f0561f44a9e5 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:50.449372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344e0de-4910-40d7-a006-25bcb5b51865 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:50.480620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:50.481017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78f0a0-30cf-4f74-96c9-f990a75a61da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:50.482626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78f0a0-30cf-4f74-96c9-f990a75a61da None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:50.483162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78f0a0-30cf-4f74-96c9-f990a75a61da None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:50.483581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78f0a0-30cf-4f74-96c9-f990a75a61da None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:50.485069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78f0a0-30cf-4f74-96c9-f990a75a61da None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:50.485662 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78f0a0-30cf-4f74-96c9-f990a75a61da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:50.487345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78f0a0-30cf-4f74-96c9-f990a75a61da None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:50.490655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1ee03-c876-415c-a4cb-92f682ed299a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:50.491677 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1ee03-c876-415c-a4cb-92f682ed299a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:50.491886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1ee03-c876-415c-a4cb-92f682ed299a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:50.492231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1ee03-c876-415c-a4cb-92f682ed299a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:50.503659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1ee03-c876-415c-a4cb-92f682ed299a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:50.505272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1ee03-c876-415c-a4cb-92f682ed299a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:50.510371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 295/584] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Jul 25 08:32:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:50.511452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78f0a0-30cf-4f74-96c9-f990a75a61da None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-11c1ee03-c876-415c-a4cb-92f682ed299a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:50.524680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:50.525785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:50.526897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:50.526897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:50.526897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:50.526897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:50.527832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 290/585] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:50.529038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1ee03-c876-415c-a4cb-92f682ed299a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:50.552260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:50.641754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78f0a0-30cf-4f74-96c9-f990a75a61da None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:50.651153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1ee03-c876-415c-a4cb-92f682ed299a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:50.679098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06cb7f-4a21-44aa-9dbc-39222dcadf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:50.680008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06cb7f-4a21-44aa-9dbc-39222dcadf34 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:50.680355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06cb7f-4a21-44aa-9dbc-39222dcadf34 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:50.680777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06cb7f-4a21-44aa-9dbc-39222dcadf34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:50.691812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06cb7f-4a21-44aa-9dbc-39222dcadf34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:50.693804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06cb7f-4a21-44aa-9dbc-39222dcadf34 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:50.696017 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fec520-563e-44c7-a816-684d181160ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:50.697812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fec520-563e-44c7-a816-684d181160ad None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:50.698241 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fec520-563e-44c7-a816-684d181160ad None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:50.698656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fec520-563e-44c7-a816-684d181160ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:50.711565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fec520-563e-44c7-a816-684d181160ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-0d06cb7f-4a21-44aa-9dbc-39222dcadf34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:50.713166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:50.714215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:50.715261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:50.715261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:50.715261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:50.715261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:50.715261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 296/586] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:50.715644 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fec520-563e-44c7-a816-684d181160ad None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:50.715813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06cb7f-4a21-44aa-9dbc-39222dcadf34 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-69fec520-563e-44c7-a816-684d181160ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:50.738757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:50.739835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:50.740946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:50.740946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:50.740946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:50.740946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:50.743654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 291/587] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:50.744873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fec520-563e-44c7-a816-684d181160ad None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:50.786579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:50.823897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:50.849516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06cb7f-4a21-44aa-9dbc-39222dcadf34 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:50.860979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fec520-563e-44c7-a816-684d181160ad None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:50.893384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f8a7d-0407-4db8-aa0e-c7c615680d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:50.894173 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f8a7d-0407-4db8-aa0e-c7c615680d4c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:50.894613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f8a7d-0407-4db8-aa0e-c7c615680d4c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:50.895121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f8a7d-0407-4db8-aa0e-c7c615680d4c None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:50.896061 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f8a7d-0407-4db8-aa0e-c7c615680d4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:50.896554 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f8a7d-0407-4db8-aa0e-c7c615680d4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:50.898125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f8a7d-0407-4db8-aa0e-c7c615680d4c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:50.898595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724035d6-30ca-4a51-8610-70c401966a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:50.899409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724035d6-30ca-4a51-8610-70c401966a45 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:50.900086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724035d6-30ca-4a51-8610-70c401966a45 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:50.900504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724035d6-30ca-4a51-8610-70c401966a45 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:50.901943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724035d6-30ca-4a51-8610-70c401966a45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:50.902395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724035d6-30ca-4a51-8610-70c401966a45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:50.904135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724035d6-30ca-4a51-8610-70c401966a45 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:50.912146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 297/588] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:50.913223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f8a7d-0407-4db8-aa0e-c7c615680d4c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:50.920284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 292/589] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:50.921317 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724035d6-30ca-4a51-8610-70c401966a45 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:51.026042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:51.035793 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f8a7d-0407-4db8-aa0e-c7c615680d4c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:51.042689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724035d6-30ca-4a51-8610-70c401966a45 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:51.078937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e9af0-5e93-4d82-b5ef-aa085c1492c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:51.079305 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f8879-e582-4d5b-8583-d65994448592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:51.079975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f8879-e582-4d5b-8583-d65994448592 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:51.080253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f8879-e582-4d5b-8583-d65994448592 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:51.080530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f8879-e582-4d5b-8583-d65994448592 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:51.080993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e9af0-5e93-4d82-b5ef-aa085c1492c0 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:51.081282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f8879-e582-4d5b-8583-d65994448592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:51.081588 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f8879-e582-4d5b-8583-d65994448592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:51.081714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e9af0-5e93-4d82-b5ef-aa085c1492c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:51.082156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e9af0-5e93-4d82-b5ef-aa085c1492c0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:51.082975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f8879-e582-4d5b-8583-d65994448592 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:51.089982 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e9af0-5e93-4d82-b5ef-aa085c1492c0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:51.091618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e9af0-5e93-4d82-b5ef-aa085c1492c0 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:51.096663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 298/590] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:51.097590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f8879-e582-4d5b-8583-d65994448592 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:51.100989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-384e9af0-5e93-4d82-b5ef-aa085c1492c0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:51.110478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:51.111635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:51.112908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:51.112908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:51.112908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:51.112908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:51.112908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:51.112908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:51.112908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:51.112908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:51.112908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 08:32:51.112908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:51.116412 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 293/591] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:51.117662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e9af0-5e93-4d82-b5ef-aa085c1492c0 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:51.235086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f8879-e582-4d5b-8583-d65994448592 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:51.236254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e9af0-5e93-4d82-b5ef-aa085c1492c0 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:51.272284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e0ec8-a6d5-4afd-83d9-848a159a111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:51.273259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e0ec8-a6d5-4afd-83d9-848a159a111e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:51.273638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e0ec8-a6d5-4afd-83d9-848a159a111e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:51.274012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e0ec8-a6d5-4afd-83d9-848a159a111e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:51.281372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e0ec8-a6d5-4afd-83d9-848a159a111e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:51.281597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915136e-a31d-46c0-9b4c-9717ffdbea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:51.282550 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915136e-a31d-46c0-9b4c-9717ffdbea47 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:51.282980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915136e-a31d-46c0-9b4c-9717ffdbea47 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:51.283221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e0ec8-a6d5-4afd-83d9-848a159a111e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:51.283565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915136e-a31d-46c0-9b4c-9717ffdbea47 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:51.292294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915136e-a31d-46c0-9b4c-9717ffdbea47 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:51.293957 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915136e-a31d-46c0-9b4c-9717ffdbea47 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-5e3e0ec8-a6d5-4afd-83d9-848a159a111e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:51.306841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:51.307917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:51.309026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:51.309026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:51.309026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:51.309026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:51.309026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:51.309026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:51.309026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:51.309026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:51.310884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 294/592] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:51.311781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e0ec8-a6d5-4afd-83d9-848a159a111e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-e915136e-a31d-46c0-9b4c-9717ffdbea47 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:51.312972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:51.313865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:51.314787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:51.314787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:51.314787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:51.314787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:51.314787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:51.314787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:51.314787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:51.314787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:51.314787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 08:32:51.314787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:51.323430 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 299/593] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:51.324500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915136e-a31d-46c0-9b4c-9717ffdbea47 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:51.327538 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:51.392275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:51.434848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e0ec8-a6d5-4afd-83d9-848a159a111e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:51.435388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915136e-a31d-46c0-9b4c-9717ffdbea47 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:51.471204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848c3c6-bcde-4441-a87e-f30e97e7a8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:51.472217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848c3c6-bcde-4441-a87e-f30e97e7a8b1 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:51.472363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15ee07-d440-4564-9fe2-fd019a2f1e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:51.473344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848c3c6-bcde-4441-a87e-f30e97e7a8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:51.473755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15ee07-d440-4564-9fe2-fd019a2f1e17 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:51.473912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848c3c6-bcde-4441-a87e-f30e97e7a8b1 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:51.474229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15ee07-d440-4564-9fe2-fd019a2f1e17 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:51.474600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15ee07-d440-4564-9fe2-fd019a2f1e17 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:51.475213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848c3c6-bcde-4441-a87e-f30e97e7a8b1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:51.475574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15ee07-d440-4564-9fe2-fd019a2f1e17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:51.475818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848c3c6-bcde-4441-a87e-f30e97e7a8b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:51.476056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15ee07-d440-4564-9fe2-fd019a2f1e17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:51.477601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15ee07-d440-4564-9fe2-fd019a2f1e17 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:51.477719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848c3c6-bcde-4441-a87e-f30e97e7a8b1 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:51.490698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 300/594] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:51.491743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15ee07-d440-4564-9fe2-fd019a2f1e17 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:51.496873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 295/595] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Jul 25 08:32:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:51.497860 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848c3c6-bcde-4441-a87e-f30e97e7a8b1 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:51.569023 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:51.598759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15ee07-d440-4564-9fe2-fd019a2f1e17 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:51.606197 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848c3c6-bcde-4441-a87e-f30e97e7a8b1 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:51.622964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:51.636642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d144991-efbc-44af-b8f0-d6df246f59eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:51.637494 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d144991-efbc-44af-b8f0-d6df246f59eb None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:51.637843 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d144991-efbc-44af-b8f0-d6df246f59eb None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:51.638245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d144991-efbc-44af-b8f0-d6df246f59eb None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:51.639194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d144991-efbc-44af-b8f0-d6df246f59eb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:51.639601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d144991-efbc-44af-b8f0-d6df246f59eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:51.641185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d144991-efbc-44af-b8f0-d6df246f59eb None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:51.645978 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869214cc-c5b1-477d-ac21-3c96965df7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:51.647063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869214cc-c5b1-477d-ac21-3c96965df7e4 None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:51.647492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869214cc-c5b1-477d-ac21-3c96965df7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:51.647911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869214cc-c5b1-477d-ac21-3c96965df7e4 None admin] PATH_INFO: `/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/925bf2da4df74e86aa84ec62a267d088/roles/45565d93c8c142e086e6b7ce62ff84e4` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:51.665222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 301/596] 10.222.0.52 () {56 vars in 1139 bytes} [Mon Jul 25 08:32:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:51.665222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d144991-efbc-44af-b8f0-d6df246f59eb None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:51.668545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869214cc-c5b1-477d-ac21-3c96965df7e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ee27c4da77c47428e02fd65e890d256, user_id=925bf2da4df74e86aa84ec62a267d088, role_id=45565d93c8c142e086e6b7ce62ff84e4)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:51.670324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869214cc-c5b1-477d-ac21-3c96965df7e4 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:51.687141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 296/597] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:32:51 2022] PUT /identity/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/925bf2da4df74e86aa84ec62a267d088/roles/45565d93c8c142e086e6b7ce62ff84e4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:51.688686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869214cc-c5b1-477d-ac21-3c96965df7e4 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:51.842302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d144991-efbc-44af-b8f0-d6df246f59eb None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:51.888202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869214cc-c5b1-477d-ac21-3c96965df7e4 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:51.890337 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2b0ef-50ec-417c-8198-521adc4e532f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:51.891148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2b0ef-50ec-417c-8198-521adc4e532f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:51.891436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2b0ef-50ec-417c-8198-521adc4e532f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:51.891714 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2b0ef-50ec-417c-8198-521adc4e532f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:51.904345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2b0ef-50ec-417c-8198-521adc4e532f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:51.905858 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2b0ef-50ec-417c-8198-521adc4e532f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:51.913282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-ffd2b0ef-50ec-417c-8198-521adc4e532f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:51.926219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:51.927355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:51.928231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:51.928231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:51.928231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:51.928231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:51.928231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 302/598] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:51.934492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2b0ef-50ec-417c-8198-521adc4e532f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:51.940280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4f2c5-f66e-48db-a1a3-2d9ef68e81fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:51.942021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4f2c5-f66e-48db-a1a3-2d9ef68e81fd None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:51.942712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4f2c5-f66e-48db-a1a3-2d9ef68e81fd None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:51.943167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4f2c5-f66e-48db-a1a3-2d9ef68e81fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:51.954402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4f2c5-f66e-48db-a1a3-2d9ef68e81fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:51.956005 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4f2c5-f66e-48db-a1a3-2d9ef68e81fd None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:51.972940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-09d4f2c5-f66e-48db-a1a3-2d9ef68e81fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:51.974372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:51.975225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:51.976135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:51.976135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:51.976135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:51.976135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:51.976135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 297/599] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:51.977515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864da5d-8f16-4006-a8ad-64c2718a7f18 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:51.977625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864da5d-8f16-4006-a8ad-64c2718a7f18 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:51.977848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864da5d-8f16-4006-a8ad-64c2718a7f18 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:51.979632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 298/600] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:51.984487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864da5d-8f16-4006-a8ad-64c2718a7f18 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:52.065700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2b0ef-50ec-417c-8198-521adc4e532f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:52.114798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c64512-ca6f-4afc-8888-8d85a937a22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:52.116772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64512-ca6f-4afc-8888-8d85a937a22a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.117218 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64512-ca6f-4afc-8888-8d85a937a22a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.117649 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64512-ca6f-4afc-8888-8d85a937a22a None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.118773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c64512-ca6f-4afc-8888-8d85a937a22a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:52.119333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c64512-ca6f-4afc-8888-8d85a937a22a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:52.119834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864da5d-8f16-4006-a8ad-64c2718a7f18 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:52.121639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c64512-ca6f-4afc-8888-8d85a937a22a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:52.140056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 303/601] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:52.141641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53136d85-560d-48f7-afc4-8b50edffb211 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.142084 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53136d85-560d-48f7-afc4-8b50edffb211 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.142541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53136d85-560d-48f7-afc4-8b50edffb211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.158884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024ba2a-6a3f-432a-a167-98567682a066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:52.159784 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024ba2a-6a3f-432a-a167-98567682a066 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.160213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024ba2a-6a3f-432a-a167-98567682a066 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.160661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024ba2a-6a3f-432a-a167-98567682a066 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.161498 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024ba2a-6a3f-432a-a167-98567682a066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:52.161949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024ba2a-6a3f-432a-a167-98567682a066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:52.163485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024ba2a-6a3f-432a-a167-98567682a066 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:52.168137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-53136d85-560d-48f7-afc4-8b50edffb211 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:52.169558 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-53136d85-560d-48f7-afc4-8b50edffb211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:52.178013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 299/602] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:52.178925 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024ba2a-6a3f-432a-a167-98567682a066 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:52.210183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:52.249912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 304/603] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:52.258105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c222522-344e-4883-97a1-0caa69c954db None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.259152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c222522-344e-4883-97a1-0caa69c954db None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.259694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c222522-344e-4883-97a1-0caa69c954db None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.262047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 305/604] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:52 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:52.263023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c222522-344e-4883-97a1-0caa69c954db None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:52.293663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024ba2a-6a3f-432a-a167-98567682a066 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:52.294223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:52.330711 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75de10-cd95-4795-8465-0a8e7ae9d5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:52.331513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75de10-cd95-4795-8465-0a8e7ae9d5fe None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.331805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75de10-cd95-4795-8465-0a8e7ae9d5fe None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.332178 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75de10-cd95-4795-8465-0a8e7ae9d5fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.339443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75de10-cd95-4795-8465-0a8e7ae9d5fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:52.340938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75de10-cd95-4795-8465-0a8e7ae9d5fe None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-3b75de10-cd95-4795-8465-0a8e7ae9d5fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:52.358443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:52.359331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:52.360239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:52.360239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:52.360239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:52.360239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:52.360239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:52.360239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:52.360239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:52.360239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:52.365114 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 300/605] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:52.366141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e746e6-d231-4f7e-9395-2eb0982c9405 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.366440 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e746e6-d231-4f7e-9395-2eb0982c9405 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.366725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e746e6-d231-4f7e-9395-2eb0982c9405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.384125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c222522-344e-4883-97a1-0caa69c954db None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:52.384309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-85e746e6-d231-4f7e-9395-2eb0982c9405 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:52.385518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-85e746e6-d231-4f7e-9395-2eb0982c9405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:52.420167 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3974e-30a7-4b21-ad59-1a651493c2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:52.421128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3974e-30a7-4b21-ad59-1a651493c2f3 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.421516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3974e-30a7-4b21-ad59-1a651493c2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.421913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3974e-30a7-4b21-ad59-1a651493c2f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.430668 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3974e-30a7-4b21-ad59-1a651493c2f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:52.432951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3974e-30a7-4b21-ad59-1a651493c2f3 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-42d3974e-30a7-4b21-ad59-1a651493c2f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:52.446911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:52.447980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:52.449256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:52.449256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:52.449256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:52.449256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:52.449256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:52.449256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:52.449256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:52.449256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:52.455060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 306/606] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:52.455060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3974e-30a7-4b21-ad59-1a651493c2f3 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:52.463988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 301/607] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:52.467091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e746e6-d231-4f7e-9395-2eb0982c9405 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:52.487980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:52.552254 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:52.572540 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3974e-30a7-4b21-ad59-1a651493c2f3 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:52.574678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e746e6-d231-4f7e-9395-2eb0982c9405 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:52.609181 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f43413-530b-4e2b-9108-4896899bb24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:52.609944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f43413-530b-4e2b-9108-4896899bb24b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.610239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f43413-530b-4e2b-9108-4896899bb24b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.610516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f43413-530b-4e2b-9108-4896899bb24b None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.611307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f43413-530b-4e2b-9108-4896899bb24b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:52.611622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f43413-530b-4e2b-9108-4896899bb24b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:52.613086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f43413-530b-4e2b-9108-4896899bb24b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:52.613965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb7922-90dc-4c77-8bd6-8b604f5881fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:52.614760 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb7922-90dc-4c77-8bd6-8b604f5881fd None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.615203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb7922-90dc-4c77-8bd6-8b604f5881fd None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.615304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb7922-90dc-4c77-8bd6-8b604f5881fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.625627 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb7922-90dc-4c77-8bd6-8b604f5881fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:52.631064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 302/608] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:52.631064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f43413-530b-4e2b-9108-4896899bb24b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-00cb7922-90dc-4c77-8bd6-8b604f5881fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:52.633442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:52.634373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 307/609] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:52.635741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb7922-90dc-4c77-8bd6-8b604f5881fd None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:52.719251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:52.740990 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f43413-530b-4e2b-9108-4896899bb24b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:52.746007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb7922-90dc-4c77-8bd6-8b604f5881fd None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:52.775924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9b1a9-c448-4d0e-97cf-63152acad933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:52.776622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9b1a9-c448-4d0e-97cf-63152acad933 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.776906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9b1a9-c448-4d0e-97cf-63152acad933 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.777184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9b1a9-c448-4d0e-97cf-63152acad933 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.777973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9b1a9-c448-4d0e-97cf-63152acad933 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:52.778551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9b1a9-c448-4d0e-97cf-63152acad933 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:52.779981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9b1a9-c448-4d0e-97cf-63152acad933 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:52.782689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151b2e4-9d24-486b-8008-a78f295aecde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:52.783490 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151b2e4-9d24-486b-8008-a78f295aecde None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.783775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151b2e4-9d24-486b-8008-a78f295aecde None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.784052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151b2e4-9d24-486b-8008-a78f295aecde None admin] PATH_INFO: `/v3/roles/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.794048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 303/610] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:52.795010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9b1a9-c448-4d0e-97cf-63152acad933 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:52.799596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151b2e4-9d24-486b-8008-a78f295aecde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-2151b2e4-9d24-486b-8008-a78f295aecde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:52.807501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:52.808454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:52.809348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 308/611] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:52.810027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151b2e4-9d24-486b-8008-a78f295aecde None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:52.813460 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:52.904122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9b1a9-c448-4d0e-97cf-63152acad933 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:52.919083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151b2e4-9d24-486b-8008-a78f295aecde None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:52.942327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b397fdd-4240-44e8-aada-2dfa5f079f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:52.943226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b397fdd-4240-44e8-aada-2dfa5f079f23 None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.943516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b397fdd-4240-44e8-aada-2dfa5f079f23 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.943794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b397fdd-4240-44e8-aada-2dfa5f079f23 None admin] PATH_INFO: `/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/84871b711bcb47efb039ce40766cf3db/roles/45565d93c8c142e086e6b7ce62ff84e4` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.955492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:52.956422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a43641-c35d-489d-81b0-8877bd53c1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:52.957803 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a43641-c35d-489d-81b0-8877bd53c1b7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:52.957999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a43641-c35d-489d-81b0-8877bd53c1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:52.958210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a43641-c35d-489d-81b0-8877bd53c1b7 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:52.959115 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a43641-c35d-489d-81b0-8877bd53c1b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:52.959518 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a43641-c35d-489d-81b0-8877bd53c1b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:52.960597 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b397fdd-4240-44e8-aada-2dfa5f079f23 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ee27c4da77c47428e02fd65e890d256, user_id=84871b711bcb47efb039ce40766cf3db, role_id=45565d93c8c142e086e6b7ce62ff84e4)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:52.961109 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a43641-c35d-489d-81b0-8877bd53c1b7 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:52.962570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b397fdd-4240-44e8-aada-2dfa5f079f23 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:52.970550 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 309/612] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:52.971364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a43641-c35d-489d-81b0-8877bd53c1b7 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:52.979140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 304/613] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:32:52 2022] PUT /identity/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/84871b711bcb47efb039ce40766cf3db/roles/45565d93c8c142e086e6b7ce62ff84e4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:52.980031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b397fdd-4240-44e8-aada-2dfa5f079f23 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:53.047765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:53.093141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a43641-c35d-489d-81b0-8877bd53c1b7 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:53.099136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b397fdd-4240-44e8-aada-2dfa5f079f23 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:53.132502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04b927-022e-4aeb-b970-315acf21b8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:53.134230 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04b927-022e-4aeb-b970-315acf21b8de None admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:53.134711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04b927-022e-4aeb-b970-315acf21b8de None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:53.135183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04b927-022e-4aeb-b970-315acf21b8de None admin] PATH_INFO: `/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/62688bc3d8a449c5bc9e0d64cd2462b2/roles/45565d93c8c142e086e6b7ce62ff84e4` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:53.137007 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3bf4c-0e01-4a8d-975a-5737ced851ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:53.137694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3bf4c-0e01-4a8d-975a-5737ced851ea None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:53.137984 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3bf4c-0e01-4a8d-975a-5737ced851ea None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:53.138266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3bf4c-0e01-4a8d-975a-5737ced851ea None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:53.139044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3bf4c-0e01-4a8d-975a-5737ced851ea None admin] RBAC: Adding query filter params (name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:53.139355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3bf4c-0e01-4a8d-975a-5737ced851ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:53.141566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3bf4c-0e01-4a8d-975a-5737ced851ea None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:53.151116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 305/614] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:53.152195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3bf4c-0e01-4a8d-975a-5737ced851ea None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:53.156093 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04b927-022e-4aeb-b970-315acf21b8de None admin] RBAC: Authorizing `identity:create_grant(project_id=2ee27c4da77c47428e02fd65e890d256, user_id=62688bc3d8a449c5bc9e0d64cd2462b2, role_id=45565d93c8c142e086e6b7ce62ff84e4)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:53.157869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04b927-022e-4aeb-b970-315acf21b8de None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:53.174125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 310/615] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:32:52 2022] PUT /identity/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/62688bc3d8a449c5bc9e0d64cd2462b2/roles/45565d93c8c142e086e6b7ce62ff84e4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:53.175266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04b927-022e-4aeb-b970-315acf21b8de None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:53.240947 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:53.310053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04b927-022e-4aeb-b970-315acf21b8de None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:53.311548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3bf4c-0e01-4a8d-975a-5737ced851ea None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:53.337059 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:53.353471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71d709-ded9-4ffb-80d9-cf1f74d083f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:53.354549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71d709-ded9-4ffb-80d9-cf1f74d083f5 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:53.354549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71d709-ded9-4ffb-80d9-cf1f74d083f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:53.354823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71d709-ded9-4ffb-80d9-cf1f74d083f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:53.366050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26911aaa-4506-490f-8b3a-645790ce2665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:53.366869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26911aaa-4506-490f-8b3a-645790ce2665 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:53.367150 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26911aaa-4506-490f-8b3a-645790ce2665 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:53.367429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26911aaa-4506-490f-8b3a-645790ce2665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:53.382306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71d709-ded9-4ffb-80d9-cf1f74d083f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:53.383786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71d709-ded9-4ffb-80d9-cf1f74d083f5 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:53.386499 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26911aaa-4506-490f-8b3a-645790ce2665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:53.387985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26911aaa-4506-490f-8b3a-645790ce2665 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-4d71d709-ded9-4ffb-80d9-cf1f74d083f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:53.412762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:53.413669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:53.414544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:53.414544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:53.414544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:53.414544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-26911aaa-4506-490f-8b3a-645790ce2665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:53.416518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:53.417358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:53.418203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:53.418203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:53.418203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:53.418203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:53.418203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 306/617] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:53.418476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 311/617] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:53.421521 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26911aaa-4506-490f-8b3a-645790ce2665 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:53.423369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71d709-ded9-4ffb-80d9-cf1f74d083f5 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:53.542967 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71d709-ded9-4ffb-80d9-cf1f74d083f5 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:53.548892 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26911aaa-4506-490f-8b3a-645790ce2665 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:53.552245 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:53.580258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6e135-e453-47c3-8b64-adff6481ac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:53.580961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6e135-e453-47c3-8b64-adff6481ac3e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:53.581251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6e135-e453-47c3-8b64-adff6481ac3e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:53.581529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6e135-e453-47c3-8b64-adff6481ac3e None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:53.582304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6e135-e453-47c3-8b64-adff6481ac3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:53.582608 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6e135-e453-47c3-8b64-adff6481ac3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:53.583985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6e135-e453-47c3-8b64-adff6481ac3e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:53.593192 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d8555-0b54-4797-b300-2fdb2bd6ffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:53.594304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d8555-0b54-4797-b300-2fdb2bd6ffb1 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:53.594738 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d8555-0b54-4797-b300-2fdb2bd6ffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:53.595147 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d8555-0b54-4797-b300-2fdb2bd6ffb1 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:53.595906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d8555-0b54-4797-b300-2fdb2bd6ffb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:53.596182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d8555-0b54-4797-b300-2fdb2bd6ffb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:53.597024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 312/618] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:53.597692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d8555-0b54-4797-b300-2fdb2bd6ffb1 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:53.602431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6e135-e453-47c3-8b64-adff6481ac3e None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:53.612659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 307/619] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:53.617526 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d8555-0b54-4797-b300-2fdb2bd6ffb1 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:53.635121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:53.717215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6e135-e453-47c3-8b64-adff6481ac3e None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:53.751698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d8555-0b54-4797-b300-2fdb2bd6ffb1 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:53.754254 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978f459-220a-4cf2-b432-d1da93a4f874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:53.755234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978f459-220a-4cf2-b432-d1da93a4f874 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:53.755672 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978f459-220a-4cf2-b432-d1da93a4f874 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:53.756127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978f459-220a-4cf2-b432-d1da93a4f874 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:53.764012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978f459-220a-4cf2-b432-d1da93a4f874 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:53.765695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978f459-220a-4cf2-b432-d1da93a4f874 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-7978f459-220a-4cf2-b432-d1da93a4f874 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:53.781870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:53.783199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:53.784322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:53.784322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:53.784322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:53.784322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:53.784322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:53.784322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:53.784322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:53.784322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:53.784322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 08:32:53.784322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:53.786517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 313/620] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:32:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:32:53.792227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978f459-220a-4cf2-b432-d1da93a4f874 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:53.804116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf16adc-5757-479c-aa51-e10e8bf84272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:53.804954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf16adc-5757-479c-aa51-e10e8bf84272 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:53.805235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf16adc-5757-479c-aa51-e10e8bf84272 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:53.805512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf16adc-5757-479c-aa51-e10e8bf84272 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:53.822867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf16adc-5757-479c-aa51-e10e8bf84272 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:53.824318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf16adc-5757-479c-aa51-e10e8bf84272 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-0bf16adc-5757-479c-aa51-e10e8bf84272 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:53.849843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:53.851028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:53.852172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:53.852172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:53.852172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:53.852172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:53.852172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:53.852172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:53.852172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:53.852172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:53.852172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:32:53.852172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:53.860739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 308/621] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Jul 25 08:32:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:53.871170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf16adc-5757-479c-aa51-e10e8bf84272 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:53.903719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:53.929213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:53.948907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978f459-220a-4cf2-b432-d1da93a4f874 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:53.991669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf16adc-5757-479c-aa51-e10e8bf84272 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:53.997731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ccd8f-dbee-41ac-82a4-ffcbd9800396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:53.998414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ccd8f-dbee-41ac-82a4-ffcbd9800396 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:53.998702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ccd8f-dbee-41ac-82a4-ffcbd9800396 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:53.998977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ccd8f-dbee-41ac-82a4-ffcbd9800396 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:54.000356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ccd8f-dbee-41ac-82a4-ffcbd9800396 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:54.000717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ccd8f-dbee-41ac-82a4-ffcbd9800396 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:54.002664 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ccd8f-dbee-41ac-82a4-ffcbd9800396 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:54.025948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 314/622] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Jul 25 08:32:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:32:54.027406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51572ce-9c77-4b31-a640-a5cf38f11d44 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:54.027792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51572ce-9c77-4b31-a640-a5cf38f11d44 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:54.028444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51572ce-9c77-4b31-a640-a5cf38f11d44 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:54.030600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 315/623] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:54.032200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51572ce-9c77-4b31-a640-a5cf38f11d44 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:54.039596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1724491-df5f-4f29-bfe7-b6ca4f769abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:54.040294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1724491-df5f-4f29-bfe7-b6ca4f769abe None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:54.040600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1724491-df5f-4f29-bfe7-b6ca4f769abe None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:54.040895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1724491-df5f-4f29-bfe7-b6ca4f769abe None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:54.041760 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1724491-df5f-4f29-bfe7-b6ca4f769abe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:54.042079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1724491-df5f-4f29-bfe7-b6ca4f769abe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:54.044199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1724491-df5f-4f29-bfe7-b6ca4f769abe None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:54.066820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 309/624] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Jul 25 08:32:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:54.068233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf71ece-a366-4631-a6bc-a52cf728d3b9 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:54.068640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf71ece-a366-4631-a6bc-a52cf728d3b9 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:54.069014 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf71ece-a366-4631-a6bc-a52cf728d3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:54.094426 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-eaf71ece-a366-4631-a6bc-a52cf728d3b9 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:54.095643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-eaf71ece-a366-4631-a6bc-a52cf728d3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:54.149111 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51572ce-9c77-4b31-a640-a5cf38f11d44 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:54.187713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84524856-ba38-4799-9ebd-5699d8a68030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:54.188759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84524856-ba38-4799-9ebd-5699d8a68030 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:54.189200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84524856-ba38-4799-9ebd-5699d8a68030 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:54.189576 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84524856-ba38-4799-9ebd-5699d8a68030 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:54.200646 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 310/625] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:54.201538 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf71ece-a366-4631-a6bc-a52cf728d3b9 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:54.207032 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84524856-ba38-4799-9ebd-5699d8a68030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:54.208490 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84524856-ba38-4799-9ebd-5699d8a68030 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:54.212685 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:54.230569 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-84524856-ba38-4799-9ebd-5699d8a68030 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:54.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:54.235229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:54.236285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:54.236285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:54.236285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:54.236285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:54.236285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 316/626] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:54.236594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84524856-ba38-4799-9ebd-5699d8a68030 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:54.343905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf71ece-a366-4631-a6bc-a52cf728d3b9 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:54.362149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84524856-ba38-4799-9ebd-5699d8a68030 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:54.389232 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecd27f-db4b-414c-92d8-6bffb55d3c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:54.389899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecd27f-db4b-414c-92d8-6bffb55d3c32 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:54.390188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecd27f-db4b-414c-92d8-6bffb55d3c32 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:54.390472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecd27f-db4b-414c-92d8-6bffb55d3c32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:54.399968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68e70c-79ec-4a3e-b938-4fb9c3223c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:54.401112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68e70c-79ec-4a3e-b938-4fb9c3223c51 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:54.401625 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68e70c-79ec-4a3e-b938-4fb9c3223c51 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:54.402118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68e70c-79ec-4a3e-b938-4fb9c3223c51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:54.412874 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68e70c-79ec-4a3e-b938-4fb9c3223c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-9f68e70c-79ec-4a3e-b938-4fb9c3223c51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:54.420849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:54.421774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:54.422574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:54.422574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:54.422574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:54.422574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:54.422574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:54.422574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:54.422574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:54.422574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:54.422574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:54.422574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:54.422574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:54.422574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:54.426954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 317/627] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:54.427945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68e70c-79ec-4a3e-b938-4fb9c3223c51 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:54.474190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecd27f-db4b-414c-92d8-6bffb55d3c32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:54.476030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecd27f-db4b-414c-92d8-6bffb55d3c32 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:54.491859 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:54.510444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:54.572092 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68e70c-79ec-4a3e-b938-4fb9c3223c51 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:54.592739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 311/628] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:54.593634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecd27f-db4b-414c-92d8-6bffb55d3c32 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:54.621420 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7df94d-e41d-4a4e-8860-d39aaf73ff6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:54.622098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7df94d-e41d-4a4e-8860-d39aaf73ff6d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:54.622390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7df94d-e41d-4a4e-8860-d39aaf73ff6d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:54.622675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7df94d-e41d-4a4e-8860-d39aaf73ff6d None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:54.623415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7df94d-e41d-4a4e-8860-d39aaf73ff6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:54.623727 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7df94d-e41d-4a4e-8860-d39aaf73ff6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:54.625615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7df94d-e41d-4a4e-8860-d39aaf73ff6d None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:54.642159 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 318/629] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:54.642996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7df94d-e41d-4a4e-8860-d39aaf73ff6d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:54.721478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecd27f-db4b-414c-92d8-6bffb55d3c32 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:54.744033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:54.772637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7df94d-e41d-4a4e-8860-d39aaf73ff6d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:54.773716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd822dfc-e32a-4d7f-9787-63aa7a0ead46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:54.774867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd822dfc-e32a-4d7f-9787-63aa7a0ead46 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:54.776188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd822dfc-e32a-4d7f-9787-63aa7a0ead46 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:54.776711 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd822dfc-e32a-4d7f-9787-63aa7a0ead46 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:54.777609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd822dfc-e32a-4d7f-9787-63aa7a0ead46 None admin] RBAC: Adding query filter params (name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:54.778377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd822dfc-e32a-4d7f-9787-63aa7a0ead46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:54.779874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd822dfc-e32a-4d7f-9787-63aa7a0ead46 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:54.791390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 312/630] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:54.792467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd822dfc-e32a-4d7f-9787-63aa7a0ead46 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:54.800123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:54.825225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d386782-b458-4226-98ea-c61109c55b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:54.826323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d386782-b458-4226-98ea-c61109c55b51 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:54.826687 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d386782-b458-4226-98ea-c61109c55b51 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:54.827461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d386782-b458-4226-98ea-c61109c55b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:54.842027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d386782-b458-4226-98ea-c61109c55b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:54.843602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d386782-b458-4226-98ea-c61109c55b51 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-8d386782-b458-4226-98ea-c61109c55b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:54.868353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:54.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:54.870578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:54.870578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:54.870578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:54.870578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:54.876321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 319/631] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:54.876321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d386782-b458-4226-98ea-c61109c55b51 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:54.930486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd822dfc-e32a-4d7f-9787-63aa7a0ead46 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:54.977258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473adc65-99aa-438f-9fb0-be255e1cded9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:54.979151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473adc65-99aa-438f-9fb0-be255e1cded9 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:54.979681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473adc65-99aa-438f-9fb0-be255e1cded9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:54.981240 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473adc65-99aa-438f-9fb0-be255e1cded9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:54.991301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473adc65-99aa-438f-9fb0-be255e1cded9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:54.991672 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d386782-b458-4226-98ea-c61109c55b51 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:54.993553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473adc65-99aa-438f-9fb0-be255e1cded9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-473adc65-99aa-438f-9fb0-be255e1cded9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:55.012937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:55.014091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:55.015204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:55.015204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:55.015204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:55.015204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:55.015204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:55.015204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:55.015204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:55.015204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:55.023397 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 313/632] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:55.023397 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473adc65-99aa-438f-9fb0-be255e1cded9 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:55.047444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5f789-9754-4abd-8624-6dc567114329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:55.048662 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5f789-9754-4abd-8624-6dc567114329 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.049131 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5f789-9754-4abd-8624-6dc567114329 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.050040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5f789-9754-4abd-8624-6dc567114329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.063722 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:55.066287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5f789-9754-4abd-8624-6dc567114329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:55.068083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5f789-9754-4abd-8624-6dc567114329 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-85a5f789-9754-4abd-8624-6dc567114329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:55.093968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:55.095053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:55.096118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:55.096118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:55.096118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:55.096118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:55.103729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 320/633] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:55.103729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5f789-9754-4abd-8624-6dc567114329 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:55.130792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:55.145496 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473adc65-99aa-438f-9fb0-be255e1cded9 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:55.188375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86d733-ca78-4997-a6b1-c02784d8c8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:55.189257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86d733-ca78-4997-a6b1-c02784d8c8b1 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.189648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86d733-ca78-4997-a6b1-c02784d8c8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.190106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86d733-ca78-4997-a6b1-c02784d8c8b1 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.190869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86d733-ca78-4997-a6b1-c02784d8c8b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:55.191323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86d733-ca78-4997-a6b1-c02784d8c8b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:55.192836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86d733-ca78-4997-a6b1-c02784d8c8b1 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:55.208607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 314/634] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:55.208607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86d733-ca78-4997-a6b1-c02784d8c8b1 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:55.242487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5f789-9754-4abd-8624-6dc567114329 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:55.283972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02ef3d-4529-4828-bd29-bec133c1b642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:55.285107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02ef3d-4529-4828-bd29-bec133c1b642 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.285540 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02ef3d-4529-4828-bd29-bec133c1b642 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.285960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02ef3d-4529-4828-bd29-bec133c1b642 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.286947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02ef3d-4529-4828-bd29-bec133c1b642 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:55.287406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02ef3d-4529-4828-bd29-bec133c1b642 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:55.289046 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02ef3d-4529-4828-bd29-bec133c1b642 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:55.308942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 321/635] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:55.310844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02ef3d-4529-4828-bd29-bec133c1b642 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:55.405788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:55.422822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:55.427509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02ef3d-4529-4828-bd29-bec133c1b642 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:55.430116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86d733-ca78-4997-a6b1-c02784d8c8b1 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:55.465013 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890b901-86d8-41ad-9982-6a780e043404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:55.465816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890b901-86d8-41ad-9982-6a780e043404 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.465998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ae537-9469-4483-8c93-b20854c7f4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:55.466114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890b901-86d8-41ad-9982-6a780e043404 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.466370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890b901-86d8-41ad-9982-6a780e043404 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.466642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ae537-9469-4483-8c93-b20854c7f4c1 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.466926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ae537-9469-4483-8c93-b20854c7f4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.467198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ae537-9469-4483-8c93-b20854c7f4c1 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.467920 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ae537-9469-4483-8c93-b20854c7f4c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:55.468387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ae537-9469-4483-8c93-b20854c7f4c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:55.470091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ae537-9469-4483-8c93-b20854c7f4c1 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:55.473997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890b901-86d8-41ad-9982-6a780e043404 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:55.475469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890b901-86d8-41ad-9982-6a780e043404 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:55.484903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 315/636] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:55.485983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ae537-9469-4483-8c93-b20854c7f4c1 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-6890b901-86d8-41ad-9982-6a780e043404 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:55.490758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:55.491715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:55.497269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:55.497269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:55.497269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:55.497269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:55.497269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:55.497269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:55.497269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:55.497269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:55.497269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 322/637] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:55.497269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45f893-ea52-488c-a060-4cb70566df1d None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.497269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45f893-ea52-488c-a060-4cb70566df1d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.497862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45f893-ea52-488c-a060-4cb70566df1d None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.499387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 323/638] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:55.500174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45f893-ea52-488c-a060-4cb70566df1d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:55.589396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ae537-9469-4483-8c93-b20854c7f4c1 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:55.602369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45f893-ea52-488c-a060-4cb70566df1d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:55.621662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eba94f-fded-40c8-a90c-5f2e52ca7bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:55.622539 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eba94f-fded-40c8-a90c-5f2e52ca7bdf None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.622816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eba94f-fded-40c8-a90c-5f2e52ca7bdf None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.623087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eba94f-fded-40c8-a90c-5f2e52ca7bdf None admin] PATH_INFO: `/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/7d6be43abd85417c9fb7bbcb36a1f77d/roles/45565d93c8c142e086e6b7ce62ff84e4` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.635998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8734392-090c-4d01-9c34-1362651c2480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:55.637158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8734392-090c-4d01-9c34-1362651c2480 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.637328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eba94f-fded-40c8-a90c-5f2e52ca7bdf None admin] RBAC: Authorizing `identity:create_grant(project_id=2ee27c4da77c47428e02fd65e890d256, user_id=7d6be43abd85417c9fb7bbcb36a1f77d, role_id=45565d93c8c142e086e6b7ce62ff84e4)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:55.637457 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8734392-090c-4d01-9c34-1362651c2480 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.637739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8734392-090c-4d01-9c34-1362651c2480 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.638744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eba94f-fded-40c8-a90c-5f2e52ca7bdf None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:55.645662 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8734392-090c-4d01-9c34-1362651c2480 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:55.647377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8734392-090c-4d01-9c34-1362651c2480 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:55.650056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:55.651591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 316/639] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:32:55 2022] PUT /identity/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/7d6be43abd85417c9fb7bbcb36a1f77d/roles/45565d93c8c142e086e6b7ce62ff84e4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:32:55.652392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eba94f-fded-40c8-a90c-5f2e52ca7bdf None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-b8734392-090c-4d01-9c34-1362651c2480 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:55.663273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:55.664313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:55.665988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:55.665988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:55.665988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:55.665988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:55.665988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:55.665988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:55.665988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:55.665988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:55.665988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 08:32:55.665988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:55.666542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:55.666617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 324/640] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:55.666617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528e8aa-79f8-4268-945d-8c31d1974a46 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.666617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528e8aa-79f8-4268-945d-8c31d1974a46 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.666617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528e8aa-79f8-4268-945d-8c31d1974a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.688407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-1528e8aa-79f8-4268-945d-8c31d1974a46 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:55.689688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1528e8aa-79f8-4268-945d-8c31d1974a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:55.766313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 325/641] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:55.767709 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cd381-5572-4f4a-9951-d56c5d2958a3 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.768005 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cd381-5572-4f4a-9951-d56c5d2958a3 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.768288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cd381-5572-4f4a-9951-d56c5d2958a3 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.769944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 326/642] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:55.770511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cd381-5572-4f4a-9951-d56c5d2958a3 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:55.791988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eba94f-fded-40c8-a90c-5f2e52ca7bdf None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:55.831449 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26121cbf-8ce7-477e-b701-1ac99e67a863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:55.832123 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26121cbf-8ce7-477e-b701-1ac99e67a863 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.832446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26121cbf-8ce7-477e-b701-1ac99e67a863 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.832809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26121cbf-8ce7-477e-b701-1ac99e67a863 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.833603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26121cbf-8ce7-477e-b701-1ac99e67a863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:55.833912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26121cbf-8ce7-477e-b701-1ac99e67a863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:55.835366 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26121cbf-8ce7-477e-b701-1ac99e67a863 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:55.847190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 317/643] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:55.848225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c932a71-ccdc-4226-8666-193d36e5fef9 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.848515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c932a71-ccdc-4226-8666-193d36e5fef9 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.848827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c932a71-ccdc-4226-8666-193d36e5fef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.866619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-8c932a71-ccdc-4226-8666-193d36e5fef9 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:55.867772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8c932a71-ccdc-4226-8666-193d36e5fef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:55.894161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cd381-5572-4f4a-9951-d56c5d2958a3 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:55.927307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:55.927480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:55.935179 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615e77c-f138-4137-a7a0-a01dbd4f8c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:55.935869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615e77c-f138-4137-a7a0-a01dbd4f8c7f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:55.936145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615e77c-f138-4137-a7a0-a01dbd4f8c7f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:55.936425 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615e77c-f138-4137-a7a0-a01dbd4f8c7f None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:55.937307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615e77c-f138-4137-a7a0-a01dbd4f8c7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:55.937617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615e77c-f138-4137-a7a0-a01dbd4f8c7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:55.944601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615e77c-f138-4137-a7a0-a01dbd4f8c7f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:55.944733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 318/644] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:55.944733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c932a71-ccdc-4226-8666-193d36e5fef9 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:55.963598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 327/645] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Jul 25 08:32:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:55.964604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615e77c-f138-4137-a7a0-a01dbd4f8c7f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:56.048183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c932a71-ccdc-4226-8666-193d36e5fef9 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:56.076509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615e77c-f138-4137-a7a0-a01dbd4f8c7f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:56.084274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fe69e-3316-4567-abae-f52a99579adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:56.085191 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fe69e-3316-4567-abae-f52a99579adc None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:56.085471 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fe69e-3316-4567-abae-f52a99579adc None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:56.085741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fe69e-3316-4567-abae-f52a99579adc None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:56.098837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fe69e-3316-4567-abae-f52a99579adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-5d6fe69e-3316-4567-abae-f52a99579adc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:56.108926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:56.109866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:56.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 319/646] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:56.111562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fe69e-3316-4567-abae-f52a99579adc None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:56.118871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc9ef5-e307-4776-9458-ad669c97712e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:56.119546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc9ef5-e307-4776-9458-ad669c97712e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:56.119832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc9ef5-e307-4776-9458-ad669c97712e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:56.120112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc9ef5-e307-4776-9458-ad669c97712e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:56.121010 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc9ef5-e307-4776-9458-ad669c97712e None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=8b476d6b82ae464b81fc8498dc0e68c7) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:56.121326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc9ef5-e307-4776-9458-ad669c97712e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:56.122754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc9ef5-e307-4776-9458-ad669c97712e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:56.136937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 328/647] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:55 2022] GET /identity/v3/role_assignments?user.id=8b476d6b82ae464b81fc8498dc0e68c7&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:56.137766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc9ef5-e307-4776-9458-ad669c97712e None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:56.177705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:56.223543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:56.242400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fe69e-3316-4567-abae-f52a99579adc None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:56.276123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc9ef5-e307-4776-9458-ad669c97712e None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:56.291715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db22bd-d226-4734-9598-ff5f38b10335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:56.293369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db22bd-d226-4734-9598-ff5f38b10335 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:56.293916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db22bd-d226-4734-9598-ff5f38b10335 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:56.294348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db22bd-d226-4734-9598-ff5f38b10335 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:56.308071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db22bd-d226-4734-9598-ff5f38b10335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-b8db22bd-d226-4734-9598-ff5f38b10335 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:56.316922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:56.317861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:56.318682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:56.318682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:56.318682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:56.318682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:56.318682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:56.318682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:56.318682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:56.318682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:56.318682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:56.318682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:56.318682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:56.318682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:56.319299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 320/648] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:56.319866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db22bd-d226-4734-9598-ff5f38b10335 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:56.326681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb4719-36cd-4bf9-9ee5-3a35c76ad922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:56.327350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4719-36cd-4bf9-9ee5-3a35c76ad922 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:56.327647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4719-36cd-4bf9-9ee5-3a35c76ad922 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:56.327927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4719-36cd-4bf9-9ee5-3a35c76ad922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:56.412859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb4719-36cd-4bf9-9ee5-3a35c76ad922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:56.414344 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb4719-36cd-4bf9-9ee5-3a35c76ad922 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:56.445567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db22bd-d226-4734-9598-ff5f38b10335 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:56.463926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:56.474868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:56.493401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fcd37-38c2-486c-9522-982bc69e4d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:56.494631 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fcd37-38c2-486c-9522-982bc69e4d98 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:56.495134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fcd37-38c2-486c-9522-982bc69e4d98 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:56.495820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fcd37-38c2-486c-9522-982bc69e4d98 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:56.497666 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fcd37-38c2-486c-9522-982bc69e4d98 None admin] RBAC: Adding query filter params (name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:56.498208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fcd37-38c2-486c-9522-982bc69e4d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:56.499764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fcd37-38c2-486c-9522-982bc69e4d98 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:56.511238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 321/649] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:56.512600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fcd37-38c2-486c-9522-982bc69e4d98 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:56.538627 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 329/650] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:56.544112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb4719-36cd-4bf9-9ee5-3a35c76ad922 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:56.631720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fcd37-38c2-486c-9522-982bc69e4d98 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:56.667775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb4719-36cd-4bf9-9ee5-3a35c76ad922 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:56.676952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b63089-49a1-426d-be56-67c71a8c7872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:56.678381 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b63089-49a1-426d-be56-67c71a8c7872 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:56.678871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b63089-49a1-426d-be56-67c71a8c7872 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:56.679765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b63089-49a1-426d-be56-67c71a8c7872 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:56.681035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b63089-49a1-426d-be56-67c71a8c7872 None admin] RBAC: Adding query filter params (name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:56.681559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b63089-49a1-426d-be56-67c71a8c7872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:56.683862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b63089-49a1-426d-be56-67c71a8c7872 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:56.695333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 322/651] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:56.696683 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b63089-49a1-426d-be56-67c71a8c7872 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:56.710607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a83d35-c4ea-492b-abd1-3e4401609fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:56.711605 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a83d35-c4ea-492b-abd1-3e4401609fe0 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:56.711965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a83d35-c4ea-492b-abd1-3e4401609fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:56.712317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a83d35-c4ea-492b-abd1-3e4401609fe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:56.724219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a83d35-c4ea-492b-abd1-3e4401609fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:56.725667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a83d35-c4ea-492b-abd1-3e4401609fe0 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-c0a83d35-c4ea-492b-abd1-3e4401609fe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:56.746009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:56.747110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:56.748223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:56.748223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:56.748223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:56.748223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:56.749117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:56.749448 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 330/652] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:56.750582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a83d35-c4ea-492b-abd1-3e4401609fe0 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:56.755631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:56.841230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b63089-49a1-426d-be56-67c71a8c7872 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:56.860488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a83d35-c4ea-492b-abd1-3e4401609fe0 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:56.888649 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b9779-4389-4356-a5b0-cfce41a4e1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:56.889961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b9779-4389-4356-a5b0-cfce41a4e1db None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:56.890420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b9779-4389-4356-a5b0-cfce41a4e1db None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:56.891084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b9779-4389-4356-a5b0-cfce41a4e1db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:56.897533 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5755521-7226-4007-b2e1-74677abed94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:56.898435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5755521-7226-4007-b2e1-74677abed94a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:56.898817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5755521-7226-4007-b2e1-74677abed94a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:56.899191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5755521-7226-4007-b2e1-74677abed94a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:56.905266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b9779-4389-4356-a5b0-cfce41a4e1db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:56.907387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b9779-4389-4356-a5b0-cfce41a4e1db None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:56.911125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5755521-7226-4007-b2e1-74677abed94a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:56.912701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5755521-7226-4007-b2e1-74677abed94a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-615b9779-4389-4356-a5b0-cfce41a4e1db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:56.928797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:56.929878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:56.931066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:56.931066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:56.931066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:56.931066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-c5755521-7226-4007-b2e1-74677abed94a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:56.932682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:56.933551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:56.934508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:56.934508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:56.934508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:56.934508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:56.935052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 323/653] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:56.935971 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b9779-4389-4356-a5b0-cfce41a4e1db None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:56.938799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 331/654] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:56.939740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5755521-7226-4007-b2e1-74677abed94a None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:57.001175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:57.048077 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:57.058195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b9779-4389-4356-a5b0-cfce41a4e1db None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:57.073869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5755521-7226-4007-b2e1-74677abed94a None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:57.092362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78170e-645a-4679-ba91-6cc48112a68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:57.093063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78170e-645a-4679-ba91-6cc48112a68b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:57.093345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78170e-645a-4679-ba91-6cc48112a68b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:57.093620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78170e-645a-4679-ba91-6cc48112a68b None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:57.094337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78170e-645a-4679-ba91-6cc48112a68b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:57.094641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78170e-645a-4679-ba91-6cc48112a68b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:57.095996 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78170e-645a-4679-ba91-6cc48112a68b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:57.108884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 324/655] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:57.109830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78170e-645a-4679-ba91-6cc48112a68b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:57.118842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847a48b-c260-478c-827c-9a4ec517aaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:57.120198 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847a48b-c260-478c-827c-9a4ec517aaa3 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:57.120746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847a48b-c260-478c-827c-9a4ec517aaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:57.121252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847a48b-c260-478c-827c-9a4ec517aaa3 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:57.122456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847a48b-c260-478c-827c-9a4ec517aaa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:57.123204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847a48b-c260-478c-827c-9a4ec517aaa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:57.125031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847a48b-c260-478c-827c-9a4ec517aaa3 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:57.141360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 332/656] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:57.142439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847a48b-c260-478c-827c-9a4ec517aaa3 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:57.218186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78170e-645a-4679-ba91-6cc48112a68b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:57.263705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cd844-d487-4fbd-894c-513e11f373b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:57.264930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847a48b-c260-478c-827c-9a4ec517aaa3 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:57.265315 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cd844-d487-4fbd-894c-513e11f373b7 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:57.265797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cd844-d487-4fbd-894c-513e11f373b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:57.266248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cd844-d487-4fbd-894c-513e11f373b7 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:57.267415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cd844-d487-4fbd-894c-513e11f373b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:57.268819 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cd844-d487-4fbd-894c-513e11f373b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:57.270900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cd844-d487-4fbd-894c-513e11f373b7 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:57.284312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:57.288638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 325/657] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:57.290040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cd844-d487-4fbd-894c-513e11f373b7 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:57.306741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:57.319562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed922916-2592-43d6-9f94-9f98207115a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:57.320661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed922916-2592-43d6-9f94-9f98207115a0 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:57.321025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed922916-2592-43d6-9f94-9f98207115a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:57.321394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed922916-2592-43d6-9f94-9f98207115a0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:57.329665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed922916-2592-43d6-9f94-9f98207115a0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:57.331337 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed922916-2592-43d6-9f94-9f98207115a0 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-ed922916-2592-43d6-9f94-9f98207115a0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:57.348504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:57.349696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:57.350857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:57.350857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:57.350857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:57.350857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:57.350857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:57.350857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:57.350857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:57.350857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:57.350857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 08:32:57.350857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:57.353099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 333/658] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:57.353987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed922916-2592-43d6-9f94-9f98207115a0 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:57.433530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cd844-d487-4fbd-894c-513e11f373b7 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:57.464467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed922916-2592-43d6-9f94-9f98207115a0 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:57.479294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6ce07-d91c-45ea-b14b-07f398d64d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:57.480807 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6ce07-d91c-45ea-b14b-07f398d64d6a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:57.481508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6ce07-d91c-45ea-b14b-07f398d64d6a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:57.481938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6ce07-d91c-45ea-b14b-07f398d64d6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:57.491601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6ce07-d91c-45ea-b14b-07f398d64d6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:57.493491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6ce07-d91c-45ea-b14b-07f398d64d6a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:57.507989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8025f0-7973-4f13-8c2b-9ca5b870a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:57.508814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8025f0-7973-4f13-8c2b-9ca5b870a720 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:57.509096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8025f0-7973-4f13-8c2b-9ca5b870a720 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:57.509370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8025f0-7973-4f13-8c2b-9ca5b870a720 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-4eb6ce07-d91c-45ea-b14b-07f398d64d6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:57.514929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:57.515811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:57.516749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:57.516749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:57.516749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:57.516749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:57.516749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:57.516749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:57.516749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:57.516749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:57.518773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8025f0-7973-4f13-8c2b-9ca5b870a720 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:57.521160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8025f0-7973-4f13-8c2b-9ca5b870a720 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:57.523206 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 326/659] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:57.524062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6ce07-d91c-45ea-b14b-07f398d64d6a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-ce8025f0-7973-4f13-8c2b-9ca5b870a720 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:57.538721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:57.539665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:57.541191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:57.541191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:57.541191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:57.541191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:57.541191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:57.541191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:57.541191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:57.541191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:57.541191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 08:32:57.541191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:57.541191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 334/660] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:57.541191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8025f0-7973-4f13-8c2b-9ca5b870a720 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:57.582733 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:57.614622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:57.656044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8025f0-7973-4f13-8c2b-9ca5b870a720 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:57.659399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6ce07-d91c-45ea-b14b-07f398d64d6a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:57.694304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9aba9-ea4b-4a55-a903-4be7cbb66334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:57.695164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9aba9-ea4b-4a55-a903-4be7cbb66334 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:57.695518 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9aba9-ea4b-4a55-a903-4be7cbb66334 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:57.695891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9aba9-ea4b-4a55-a903-4be7cbb66334 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:57.696954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9aba9-ea4b-4a55-a903-4be7cbb66334 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:57.697362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9aba9-ea4b-4a55-a903-4be7cbb66334 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:57.698991 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9aba9-ea4b-4a55-a903-4be7cbb66334 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:57.706173 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad08b60-849a-450e-bc24-3f1fdcbf1fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:57.708213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad08b60-849a-450e-bc24-3f1fdcbf1fd2 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:57.708705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad08b60-849a-450e-bc24-3f1fdcbf1fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:57.709126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad08b60-849a-450e-bc24-3f1fdcbf1fd2 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:57.710916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad08b60-849a-450e-bc24-3f1fdcbf1fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:57.711388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad08b60-849a-450e-bc24-3f1fdcbf1fd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:57.713059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad08b60-849a-450e-bc24-3f1fdcbf1fd2 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:57.717968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 335/661] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:57.718867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9aba9-ea4b-4a55-a903-4be7cbb66334 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:57.734066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 327/662] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Jul 25 08:32:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:57.735693 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd5330-fd6b-4b76-93f5-3d52ab26280c None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:57.736131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd5330-fd6b-4b76-93f5-3d52ab26280c None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:57.736552 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd5330-fd6b-4b76-93f5-3d52ab26280c None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:57.738005 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 328/663] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:57.738997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd5330-fd6b-4b76-93f5-3d52ab26280c None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:57.827017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:57.834175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9aba9-ea4b-4a55-a903-4be7cbb66334 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:57.844587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd5330-fd6b-4b76-93f5-3d52ab26280c None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:57.870401 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022bdde-29d5-4fd3-a72b-257574e1808a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:57.871082 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022bdde-29d5-4fd3-a72b-257574e1808a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:57.871365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022bdde-29d5-4fd3-a72b-257574e1808a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:57.871647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022bdde-29d5-4fd3-a72b-257574e1808a None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:57.872484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022bdde-29d5-4fd3-a72b-257574e1808a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:57.872868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022bdde-29d5-4fd3-a72b-257574e1808a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:57.873969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:57.874278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022bdde-29d5-4fd3-a72b-257574e1808a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:57.883705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da230a61-5e12-410d-abe9-f8edd413bb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:57.884360 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da230a61-5e12-410d-abe9-f8edd413bb76 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:57.884672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da230a61-5e12-410d-abe9-f8edd413bb76 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:57.885714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da230a61-5e12-410d-abe9-f8edd413bb76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:57.885827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da230a61-5e12-410d-abe9-f8edd413bb76 None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=925bf2da4df74e86aa84ec62a267d088) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:57.886065 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da230a61-5e12-410d-abe9-f8edd413bb76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:57.887473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da230a61-5e12-410d-abe9-f8edd413bb76 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:57.898376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 329/664] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:57 2022] GET /identity/v3/role_assignments?user.id=925bf2da4df74e86aa84ec62a267d088&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:57.898376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da230a61-5e12-410d-abe9-f8edd413bb76 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:57.898818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 336/665] 10.222.0.52 () {56 vars in 1139 bytes} [Mon Jul 25 08:32:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:57.900272 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f98e6-9593-4fdc-b59d-6d630ba21dba None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:57.900594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f98e6-9593-4fdc-b59d-6d630ba21dba None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:57.900880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f98e6-9593-4fdc-b59d-6d630ba21dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:57.921525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-658f98e6-9593-4fdc-b59d-6d630ba21dba None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:57.922772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-658f98e6-9593-4fdc-b59d-6d630ba21dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:58.013011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 337/666] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:58.014080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f98e6-9593-4fdc-b59d-6d630ba21dba None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:58.031415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da230a61-5e12-410d-abe9-f8edd413bb76 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:58.083905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2592f48-8c50-48df-ab1a-b9d6ba757651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:58.085180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2592f48-8c50-48df-ab1a-b9d6ba757651 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:58.085180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2592f48-8c50-48df-ab1a-b9d6ba757651 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:58.085351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2592f48-8c50-48df-ab1a-b9d6ba757651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:58.131436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:58.152536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:58.154239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f98e6-9593-4fdc-b59d-6d630ba21dba None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:58.172380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2592f48-8c50-48df-ab1a-b9d6ba757651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:58.174127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2592f48-8c50-48df-ab1a-b9d6ba757651 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:58.201933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f416b-7813-440e-a446-4462c5ed1ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:58.202901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f416b-7813-440e-a446-4462c5ed1ff5 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:58.203285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f416b-7813-440e-a446-4462c5ed1ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:58.204025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f416b-7813-440e-a446-4462c5ed1ff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:58.292140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f416b-7813-440e-a446-4462c5ed1ff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:58.293684 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f416b-7813-440e-a446-4462c5ed1ff5 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:58.301812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 330/667] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:58.301812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2592f48-8c50-48df-ab1a-b9d6ba757651 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:58.387851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:58.400284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:58.410258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2592f48-8c50-48df-ab1a-b9d6ba757651 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:58.412792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 338/668] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:32:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:32:58.414149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9ee1a-1414-4982-9a4c-e13325fb3aa5 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:58.414437 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9ee1a-1414-4982-9a4c-e13325fb3aa5 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:58.414715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9ee1a-1414-4982-9a4c-e13325fb3aa5 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:58.419994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 339/669] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:32:58.419994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9ee1a-1414-4982-9a4c-e13325fb3aa5 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:58.448168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b8ee7-15db-4351-867d-6038956e0ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:58.449029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b8ee7-15db-4351-867d-6038956e0ace None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:58.449316 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b8ee7-15db-4351-867d-6038956e0ace None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:58.449598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b8ee7-15db-4351-867d-6038956e0ace None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:58.460860 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b8ee7-15db-4351-867d-6038956e0ace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-701b8ee7-15db-4351-867d-6038956e0ace None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:58.468684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:58.469624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:58.470453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 331/670] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:32:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:58.471136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b8ee7-15db-4351-867d-6038956e0ace None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:58.535304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9ee1a-1414-4982-9a4c-e13325fb3aa5 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:58.572619 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29135f-ef41-4c43-b4ad-ec5f85dd2f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:58.573424 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29135f-ef41-4c43-b4ad-ec5f85dd2f95 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:58.573711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29135f-ef41-4c43-b4ad-ec5f85dd2f95 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:58.573993 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29135f-ef41-4c43-b4ad-ec5f85dd2f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:58.574216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b8ee7-15db-4351-867d-6038956e0ace None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:58.585483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29135f-ef41-4c43-b4ad-ec5f85dd2f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:58.586933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29135f-ef41-4c43-b4ad-ec5f85dd2f95 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-ab29135f-ef41-4c43-b4ad-ec5f85dd2f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:58.605024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:58.605943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:58.606814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:58.606814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:58.606814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:58.606814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:58.606814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 340/671] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:58.607722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb478b-ebbc-4a31-a560-bd0f70be8f1e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:58.608012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb478b-ebbc-4a31-a560-bd0f70be8f1e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:58.608292 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb478b-ebbc-4a31-a560-bd0f70be8f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:58.612434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c25424-16d7-4a12-bd31-39e2d4c83ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:58.613286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c25424-16d7-4a12-bd31-39e2d4c83ec0 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:58.613569 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c25424-16d7-4a12-bd31-39e2d4c83ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:58.613840 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c25424-16d7-4a12-bd31-39e2d4c83ec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:58.625384 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c25424-16d7-4a12-bd31-39e2d4c83ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:58.625996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-fdeb478b-ebbc-4a31-a560-bd0f70be8f1e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:32:58.626828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c25424-16d7-4a12-bd31-39e2d4c83ec0 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:58.627228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fdeb478b-ebbc-4a31-a560-bd0f70be8f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-07c25424-16d7-4a12-bd31-39e2d4c83ec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:58.646099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:58.646993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:58.647870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:58.647870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:58.647870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:58.647870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:58.647870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 332/672] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:58.648148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c25424-16d7-4a12-bd31-39e2d4c83ec0 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:58.656310 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:58.661340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:58.708739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 341/673] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:32:58.708739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb478b-ebbc-4a31-a560-bd0f70be8f1e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:58.768255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c25424-16d7-4a12-bd31-39e2d4c83ec0 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:58.803178 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c524771-4ff6-45ab-bf5d-b2d106691af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:58.803843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c524771-4ff6-45ab-bf5d-b2d106691af5 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:58.804124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c524771-4ff6-45ab-bf5d-b2d106691af5 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:58.804397 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c524771-4ff6-45ab-bf5d-b2d106691af5 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:58.805184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c524771-4ff6-45ab-bf5d-b2d106691af5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:58.805488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c524771-4ff6-45ab-bf5d-b2d106691af5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:58.806849 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c524771-4ff6-45ab-bf5d-b2d106691af5 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:58.820559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 333/674] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:32:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:58.820559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c524771-4ff6-45ab-bf5d-b2d106691af5 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:58.820830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb478b-ebbc-4a31-a560-bd0f70be8f1e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:58.854774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d68759-8e63-4fce-8bd1-d2c1043522c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:58.855780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d68759-8e63-4fce-8bd1-d2c1043522c2 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:58.855780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d68759-8e63-4fce-8bd1-d2c1043522c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:58.856074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d68759-8e63-4fce-8bd1-d2c1043522c2 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:58.856749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d68759-8e63-4fce-8bd1-d2c1043522c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:58.857063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d68759-8e63-4fce-8bd1-d2c1043522c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:58.858436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d68759-8e63-4fce-8bd1-d2c1043522c2 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:58.871119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 342/675] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:58.871836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d68759-8e63-4fce-8bd1-d2c1043522c2 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:58.891811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:58.901756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:58.930515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c524771-4ff6-45ab-bf5d-b2d106691af5 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:58.966563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975b808-69dd-41f0-9b66-89582d0295ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:58.967242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975b808-69dd-41f0-9b66-89582d0295ca None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:58.967542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975b808-69dd-41f0-9b66-89582d0295ca None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:58.967780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975b808-69dd-41f0-9b66-89582d0295ca None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:58.968674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975b808-69dd-41f0-9b66-89582d0295ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:58.968991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975b808-69dd-41f0-9b66-89582d0295ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:58.970486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975b808-69dd-41f0-9b66-89582d0295ca None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:58.980214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d68759-8e63-4fce-8bd1-d2c1043522c2 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:58.984615 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 334/676] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:58.985672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975b808-69dd-41f0-9b66-89582d0295ca None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:59.015883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ca935-a145-4840-b78c-59789c45117f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:59.016715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ca935-a145-4840-b78c-59789c45117f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:59.017019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ca935-a145-4840-b78c-59789c45117f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:59.017283 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ca935-a145-4840-b78c-59789c45117f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:59.027739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ca935-a145-4840-b78c-59789c45117f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-dc5ca935-a145-4840-b78c-59789c45117f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:32:59.036161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:59.037169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:59.038060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 343/677] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:32:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:59.038790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ca935-a145-4840-b78c-59789c45117f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:59.099315 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975b808-69dd-41f0-9b66-89582d0295ca None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:59.127361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:59.133728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:59.138416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52f7c3-becb-4696-a0a8-8d248cc9986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:59.139276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52f7c3-becb-4696-a0a8-8d248cc9986a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:59.139558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52f7c3-becb-4696-a0a8-8d248cc9986a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:59.139831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52f7c3-becb-4696-a0a8-8d248cc9986a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:59.149804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ca935-a145-4840-b78c-59789c45117f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:59.150358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52f7c3-becb-4696-a0a8-8d248cc9986a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:59.151858 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52f7c3-becb-4696-a0a8-8d248cc9986a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-cb52f7c3-becb-4696-a0a8-8d248cc9986a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:59.169993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:59.170938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:59.171830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:59.171830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:59.171830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:59.171830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:59.171830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 335/678] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:59.172111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52f7c3-becb-4696-a0a8-8d248cc9986a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:59.193286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f53d79-1b9c-4230-9f64-fe44b92650e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:59.194096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f53d79-1b9c-4230-9f64-fe44b92650e7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:59.194383 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f53d79-1b9c-4230-9f64-fe44b92650e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:59.194663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f53d79-1b9c-4230-9f64-fe44b92650e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:59.201833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f53d79-1b9c-4230-9f64-fe44b92650e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:59.203289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f53d79-1b9c-4230-9f64-fe44b92650e7 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-20f53d79-1b9c-4230-9f64-fe44b92650e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:59.219744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:59.220653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:59.221525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:59.221525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:59.221525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:59.221525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:59.221525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:59.221525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:59.221525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:59.221525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:59.221525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 344/679] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:59.221998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f53d79-1b9c-4230-9f64-fe44b92650e7 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:59.285731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52f7c3-becb-4696-a0a8-8d248cc9986a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:59.326250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c65086-5b60-4d3f-a405-2a530b810d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:59.327718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c65086-5b60-4d3f-a405-2a530b810d94 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:59.328270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c65086-5b60-4d3f-a405-2a530b810d94 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:59.328801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c65086-5b60-4d3f-a405-2a530b810d94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:59.336107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c65086-5b60-4d3f-a405-2a530b810d94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:59.337952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c65086-5b60-4d3f-a405-2a530b810d94 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:59.339856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f53d79-1b9c-4230-9f64-fe44b92650e7 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-c6c65086-5b60-4d3f-a405-2a530b810d94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:32:59.354751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:59.355841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:59.356992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:32:59.356992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:32:59.356992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:32:59.356992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:32:59.356992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:32:59.356992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:32:59.356992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:32:59.356992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:59.358742 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 336/680] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:32:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:59.359643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c65086-5b60-4d3f-a405-2a530b810d94 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:59.379064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23ec2b-c466-4f7c-b484-3a85ea558395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:59.379815 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23ec2b-c466-4f7c-b484-3a85ea558395 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:59.380106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23ec2b-c466-4f7c-b484-3a85ea558395 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:59.380385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23ec2b-c466-4f7c-b484-3a85ea558395 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:59.381157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23ec2b-c466-4f7c-b484-3a85ea558395 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:59.381487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23ec2b-c466-4f7c-b484-3a85ea558395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:59.382816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23ec2b-c466-4f7c-b484-3a85ea558395 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:59.396046 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 345/681] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:32:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:59.396781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23ec2b-c466-4f7c-b484-3a85ea558395 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:59.405089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:59.408400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:59.466715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c65086-5b60-4d3f-a405-2a530b810d94 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:59.502503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28e441-dd53-4638-bc92-6eef35bdabd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:59.502732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23ec2b-c466-4f7c-b484-3a85ea558395 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:59.503171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e441-dd53-4638-bc92-6eef35bdabd3 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:59.503480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e441-dd53-4638-bc92-6eef35bdabd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:59.503755 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e441-dd53-4638-bc92-6eef35bdabd3 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:59.504472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28e441-dd53-4638-bc92-6eef35bdabd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:59.504803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28e441-dd53-4638-bc92-6eef35bdabd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:59.506158 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28e441-dd53-4638-bc92-6eef35bdabd3 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:59.517719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 337/682] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:59.518342 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28e441-dd53-4638-bc92-6eef35bdabd3 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:59.545913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd39c4-b6e5-4ec5-83f7-e4d63f028e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:59.546661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd39c4-b6e5-4ec5-83f7-e4d63f028e08 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:59.546849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd39c4-b6e5-4ec5-83f7-e4d63f028e08 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:59.547122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd39c4-b6e5-4ec5-83f7-e4d63f028e08 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:59.547981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd39c4-b6e5-4ec5-83f7-e4d63f028e08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:59.548261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd39c4-b6e5-4ec5-83f7-e4d63f028e08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:59.549776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd39c4-b6e5-4ec5-83f7-e4d63f028e08 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:59.567247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 346/683] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:59.567247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd39c4-b6e5-4ec5-83f7-e4d63f028e08 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:59.623111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:59.627091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28e441-dd53-4638-bc92-6eef35bdabd3 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:59.640885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:59.663311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97419692-b5f6-4df1-bc01-5c7bb21abe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:59.663907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97419692-b5f6-4df1-bc01-5c7bb21abe24 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:59.664193 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97419692-b5f6-4df1-bc01-5c7bb21abe24 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:59.664473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97419692-b5f6-4df1-bc01-5c7bb21abe24 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:59.665298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97419692-b5f6-4df1-bc01-5c7bb21abe24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:59.665612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97419692-b5f6-4df1-bc01-5c7bb21abe24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:59.667045 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97419692-b5f6-4df1-bc01-5c7bb21abe24 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:59.678316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd39c4-b6e5-4ec5-83f7-e4d63f028e08 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:59.683821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 338/684] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:32:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:59.683821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97419692-b5f6-4df1-bc01-5c7bb21abe24 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:59.719702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f64e2a-9e57-4524-a39d-c40c0439234f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:59.720498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f64e2a-9e57-4524-a39d-c40c0439234f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:59.720832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f64e2a-9e57-4524-a39d-c40c0439234f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:59.721104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f64e2a-9e57-4524-a39d-c40c0439234f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:59.733033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f64e2a-9e57-4524-a39d-c40c0439234f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:59.734467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f64e2a-9e57-4524-a39d-c40c0439234f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-47f64e2a-9e57-4524-a39d-c40c0439234f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:32:59.753872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:59.754960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:59.755850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:32:59.755850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:32:59.755850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:32:59.755850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:32:59.755850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 347/685] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:59.755850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f64e2a-9e57-4524-a39d-c40c0439234f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:59.795236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97419692-b5f6-4df1-bc01-5c7bb21abe24 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:59.830973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498277a-7751-4eb2-b889-9c291a0a197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:59.832799 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498277a-7751-4eb2-b889-9c291a0a197e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:59.833086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498277a-7751-4eb2-b889-9c291a0a197e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:59.833365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498277a-7751-4eb2-b889-9c291a0a197e None admin] PATH_INFO: `/v3/users/swift` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:59.840735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498277a-7751-4eb2-b889-9c291a0a197e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:59.842182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498277a-7751-4eb2-b889-9c291a0a197e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-9498277a-7751-4eb2-b889-9c291a0a197e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:32:59.856808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:32:59.857764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:32:59.858712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:32:59.858712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:32:59.858712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:32:59.858712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:32:59.858712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:32:59.858712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:32:59.858712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:32:59.858712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:32:59.858712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 08:32:59.858712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:32:59.858712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 339/686] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:32:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:32:59.859323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498277a-7751-4eb2-b889-9c291a0a197e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:59.869407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f64e2a-9e57-4524-a39d-c40c0439234f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:32:59.878966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:59.896474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:32:59.916289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee0842-c992-4d96-9e38-09784efd769c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:32:59.917178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee0842-c992-4d96-9e38-09784efd769c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:32:59.917464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee0842-c992-4d96-9e38-09784efd769c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:32:59.917738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee0842-c992-4d96-9e38-09784efd769c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:32:59.918861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee0842-c992-4d96-9e38-09784efd769c None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=84871b711bcb47efb039ce40766cf3db) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:32:59.919549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee0842-c992-4d96-9e38-09784efd769c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:32:59.921528 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee0842-c992-4d96-9e38-09784efd769c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:32:59.929696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 348/687] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:59 2022] GET /identity/v3/role_assignments?user.id=84871b711bcb47efb039ce40766cf3db&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:32:59.930580 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee0842-c992-4d96-9e38-09784efd769c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:32:59.988214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498277a-7751-4eb2-b889-9c291a0a197e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:00.040332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443d6fd-d7b9-4b8d-a5bc-3b75a9f77cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:00.041877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443d6fd-d7b9-4b8d-a5bc-3b75a9f77cf4 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:00.042338 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443d6fd-d7b9-4b8d-a5bc-3b75a9f77cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:00.042769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443d6fd-d7b9-4b8d-a5bc-3b75a9f77cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:00.044450 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443d6fd-d7b9-4b8d-a5bc-3b75a9f77cf4 None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=62688bc3d8a449c5bc9e0d64cd2462b2) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:00.044976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443d6fd-d7b9-4b8d-a5bc-3b75a9f77cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:00.047446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443d6fd-d7b9-4b8d-a5bc-3b75a9f77cf4 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:00.056601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 340/688] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:32:59 2022] GET /identity/v3/role_assignments?user.id=62688bc3d8a449c5bc9e0d64cd2462b2&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:00.058509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443d6fd-d7b9-4b8d-a5bc-3b75a9f77cf4 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:00.072098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee0842-c992-4d96-9e38-09784efd769c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:00.116413 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f71f6-8b96-46af-a636-5a3e272ecb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:00.117445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f71f6-8b96-46af-a636-5a3e272ecb8c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:00.117887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f71f6-8b96-46af-a636-5a3e272ecb8c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:00.118325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f71f6-8b96-46af-a636-5a3e272ecb8c None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:00.119255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f71f6-8b96-46af-a636-5a3e272ecb8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:00.119748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f71f6-8b96-46af-a636-5a3e272ecb8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:00.121375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f71f6-8b96-46af-a636-5a3e272ecb8c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:00.135455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 349/689] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:00.137261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4e9ba-a954-49b6-b9e5-167c33675b8c None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:00.137555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4e9ba-a954-49b6-b9e5-167c33675b8c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:00.137839 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4e9ba-a954-49b6-b9e5-167c33675b8c None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:00.139426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 350/690] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:00.144752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4e9ba-a954-49b6-b9e5-167c33675b8c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:00.176376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:00.203104 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443d6fd-d7b9-4b8d-a5bc-3b75a9f77cf4 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:00.203393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:00.246831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7553b9a-55c2-4b15-9ab1-62a3a1e7ef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:00.247521 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7553b9a-55c2-4b15-9ab1-62a3a1e7ef63 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:00.247805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7553b9a-55c2-4b15-9ab1-62a3a1e7ef63 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:00.248083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7553b9a-55c2-4b15-9ab1-62a3a1e7ef63 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:00.249044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7553b9a-55c2-4b15-9ab1-62a3a1e7ef63 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:00.249644 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7553b9a-55c2-4b15-9ab1-62a3a1e7ef63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:00.251135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7553b9a-55c2-4b15-9ab1-62a3a1e7ef63 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:00.272325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 341/691] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Jul 25 08:33:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:00.273794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa1baa-7449-4f56-b056-2f5575508130 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:00.274079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa1baa-7449-4f56-b056-2f5575508130 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:00.274352 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa1baa-7449-4f56-b056-2f5575508130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:00.276543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4e9ba-a954-49b6-b9e5-167c33675b8c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:00.296800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-0cfa1baa-7449-4f56-b056-2f5575508130 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:00.298331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0cfa1baa-7449-4f56-b056-2f5575508130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:00.326432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d460e5-64e3-494f-b3d8-e0cbb38ec55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:00.329535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d460e5-64e3-494f-b3d8-e0cbb38ec55d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:00.329535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d460e5-64e3-494f-b3d8-e0cbb38ec55d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:00.329535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d460e5-64e3-494f-b3d8-e0cbb38ec55d None admin] PATH_INFO: `/v3/users/nova` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:00.341241 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d460e5-64e3-494f-b3d8-e0cbb38ec55d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:00.343628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d460e5-64e3-494f-b3d8-e0cbb38ec55d None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-e6d460e5-64e3-494f-b3d8-e0cbb38ec55d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:00.364393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:00.365479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:00.366675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:00.366675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:00.366675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:00.366675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:00.366675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:00.366675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:00.366675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:00.366675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:00.366675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:33:00.366675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:00.366675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 351/692] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Jul 25 08:33:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:00.367743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d460e5-64e3-494f-b3d8-e0cbb38ec55d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:00.380846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 342/693] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:33:00.381888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa1baa-7449-4f56-b056-2f5575508130 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:00.479431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:00.501110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d460e5-64e3-494f-b3d8-e0cbb38ec55d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:00.504029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:00.517715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa1baa-7449-4f56-b056-2f5575508130 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:00.537738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d1cc2-9eac-4b27-9684-6a6329ee4fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:00.538394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d1cc2-9eac-4b27-9684-6a6329ee4fab None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:00.538674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d1cc2-9eac-4b27-9684-6a6329ee4fab None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:00.538948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d1cc2-9eac-4b27-9684-6a6329ee4fab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:00.557419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652e2fb-0de1-4781-99dc-a979b96fa5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:00.558205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652e2fb-0de1-4781-99dc-a979b96fa5d9 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:00.558617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652e2fb-0de1-4781-99dc-a979b96fa5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:00.558903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652e2fb-0de1-4781-99dc-a979b96fa5d9 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:00.559757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652e2fb-0de1-4781-99dc-a979b96fa5d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:00.560071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652e2fb-0de1-4781-99dc-a979b96fa5d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:00.561692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652e2fb-0de1-4781-99dc-a979b96fa5d9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:00.581376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 343/694] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Jul 25 08:33:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:00.582430 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652e2fb-0de1-4781-99dc-a979b96fa5d9 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:00.608849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d1cc2-9eac-4b27-9684-6a6329ee4fab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:00.610314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d1cc2-9eac-4b27-9684-6a6329ee4fab None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:00.697325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652e2fb-0de1-4781-99dc-a979b96fa5d9 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:00.726133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:00.735868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0258a-2c02-4896-99ba-f5fa63356409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:00.736726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0258a-2c02-4896-99ba-f5fa63356409 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:00.737086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0258a-2c02-4896-99ba-f5fa63356409 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:00.737432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0258a-2c02-4896-99ba-f5fa63356409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:00.739132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 352/695] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:33:00.739982 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d1cc2-9eac-4b27-9684-6a6329ee4fab None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:00.748935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:00.759053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0258a-2c02-4896-99ba-f5fa63356409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:00.761057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0258a-2c02-4896-99ba-f5fa63356409 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-7ab0258a-2c02-4896-99ba-f5fa63356409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:00.785999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:00.787299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:00.789121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:00.789121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:00.789121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:00.789121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:00.789121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 344/696] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:00.789121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0258a-2c02-4896-99ba-f5fa63356409 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:00.868084 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d1cc2-9eac-4b27-9684-6a6329ee4fab None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:00.908425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0258a-2c02-4896-99ba-f5fa63356409 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:00.915041 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37f8b6-5d40-4df1-8490-a788d7c9d782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:00.915939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37f8b6-5d40-4df1-8490-a788d7c9d782 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:00.916948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37f8b6-5d40-4df1-8490-a788d7c9d782 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:00.917431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37f8b6-5d40-4df1-8490-a788d7c9d782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:00.945040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2632ed-cdf5-4ffd-9e04-18852e9d4d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:00.945917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2632ed-cdf5-4ffd-9e04-18852e9d4d37 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:00.946536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2632ed-cdf5-4ffd-9e04-18852e9d4d37 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:00.946536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2632ed-cdf5-4ffd-9e04-18852e9d4d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:00.958604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2632ed-cdf5-4ffd-9e04-18852e9d4d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:00.960056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2632ed-cdf5-4ffd-9e04-18852e9d4d37 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-4f2632ed-cdf5-4ffd-9e04-18852e9d4d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:00.980934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:00.981924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:00.983203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:00.983203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:00.983203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:00.983203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:00.983203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 345/697] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:00.984462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2632ed-cdf5-4ffd-9e04-18852e9d4d37 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:01.015798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37f8b6-5d40-4df1-8490-a788d7c9d782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:01.018156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37f8b6-5d40-4df1-8490-a788d7c9d782 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:01.020246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:01.023429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:01.100959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2632ed-cdf5-4ffd-9e04-18852e9d4d37 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:01.138673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e0576-73e5-4991-b038-0a4904345ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:01.139360 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e0576-73e5-4991-b038-0a4904345ddb None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:01.139640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e0576-73e5-4991-b038-0a4904345ddb None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:01.139925 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e0576-73e5-4991-b038-0a4904345ddb None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:01.140718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e0576-73e5-4991-b038-0a4904345ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:01.141044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e0576-73e5-4991-b038-0a4904345ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:01.142639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e0576-73e5-4991-b038-0a4904345ddb None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:01.149849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 353/698] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:33:01.151012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37f8b6-5d40-4df1-8490-a788d7c9d782 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:01.161895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 346/699] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:01.162790 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e0576-73e5-4991-b038-0a4904345ddb None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:01.261299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37f8b6-5d40-4df1-8490-a788d7c9d782 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:01.262827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:01.269037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:01.273585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e0576-73e5-4991-b038-0a4904345ddb None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:01.318591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d84be-c270-47c3-9716-a1dd4fac2f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:01.319727 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d84be-c270-47c3-9716-a1dd4fac2f76 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:01.320391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d84be-c270-47c3-9716-a1dd4fac2f76 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:01.320873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d84be-c270-47c3-9716-a1dd4fac2f76 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:01.322459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d84be-c270-47c3-9716-a1dd4fac2f76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:01.322926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d84be-c270-47c3-9716-a1dd4fac2f76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:01.324430 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d84be-c270-47c3-9716-a1dd4fac2f76 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:01.326529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815b510-e57b-43c3-8265-0c908cf844d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:01.327529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815b510-e57b-43c3-8265-0c908cf844d4 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:01.327810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815b510-e57b-43c3-8265-0c908cf844d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:01.328183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815b510-e57b-43c3-8265-0c908cf844d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:01.339463 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815b510-e57b-43c3-8265-0c908cf844d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:01.340523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 354/700] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:01.341299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815b510-e57b-43c3-8265-0c908cf844d4 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:01.341681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d84be-c270-47c3-9716-a1dd4fac2f76 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-e815b510-e57b-43c3-8265-0c908cf844d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:01.359960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:01.361074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:01.362186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:01.362186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:01.362186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:01.362186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:01.367974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 347/701] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:01.367974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815b510-e57b-43c3-8265-0c908cf844d4 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:01.474071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815b510-e57b-43c3-8265-0c908cf844d4 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:01.478310 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d84be-c270-47c3-9716-a1dd4fac2f76 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:01.511842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b73487-edbd-4aaf-bce2-b771e3b8b680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:01.512811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b73487-edbd-4aaf-bce2-b771e3b8b680 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:01.513199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b73487-edbd-4aaf-bce2-b771e3b8b680 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:01.513577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b73487-edbd-4aaf-bce2-b771e3b8b680 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:01.521022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b73487-edbd-4aaf-bce2-b771e3b8b680 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:01.522534 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b73487-edbd-4aaf-bce2-b771e3b8b680 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:01.523917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f7f0e-66c0-40ed-823c-f647f529730c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:01.525027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f7f0e-66c0-40ed-823c-f647f529730c None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:01.525829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f7f0e-66c0-40ed-823c-f647f529730c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:01.526258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f7f0e-66c0-40ed-823c-f647f529730c None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:01.527122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f7f0e-66c0-40ed-823c-f647f529730c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:01.528973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f7f0e-66c0-40ed-823c-f647f529730c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-96b73487-edbd-4aaf-bce2-b771e3b8b680 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:01.538717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:01.539794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:01.540897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:01.540897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:01.540897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:01.540897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:01.540897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:01.540897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:01.540897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:01.540897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:01.542922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 348/702] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:01.543751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b73487-edbd-4aaf-bce2-b771e3b8b680 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:01.549481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:01.561277 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 355/703] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:33:01 2022] POST /identity/v3/users => generated 254 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:01.562728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f7f0e-66c0-40ed-823c-f647f529730c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:01.594498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:01.676218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b73487-edbd-4aaf-bce2-b771e3b8b680 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:01.690975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f7f0e-66c0-40ed-823c-f647f529730c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:01.716212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5282c-b588-4443-a345-c075f9967dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:01.717322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5282c-b588-4443-a345-c075f9967dfe None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:01.717784 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5282c-b588-4443-a345-c075f9967dfe None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:01.718199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5282c-b588-4443-a345-c075f9967dfe None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:01.719184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5282c-b588-4443-a345-c075f9967dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:01.719756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5282c-b588-4443-a345-c075f9967dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:01.721500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5282c-b588-4443-a345-c075f9967dfe None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:01.735827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 349/704] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:01.736228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822c49b-b6f7-43da-abdd-5953265c1971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:01.741989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822c49b-b6f7-43da-abdd-5953265c1971 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:01.741989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822c49b-b6f7-43da-abdd-5953265c1971 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:01.741989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822c49b-b6f7-43da-abdd-5953265c1971 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:01.741989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2822c49b-b6f7-43da-abdd-5953265c1971 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:01.741989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2822c49b-b6f7-43da-abdd-5953265c1971 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:01.741989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2822c49b-b6f7-43da-abdd-5953265c1971 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:01.754451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5282c-b588-4443-a345-c075f9967dfe None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:01.760455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 356/705] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:01.765879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822c49b-b6f7-43da-abdd-5953265c1971 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:01.841121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:01.888121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5282c-b588-4443-a345-c075f9967dfe None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:01.936689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd30a32-e1dc-48d8-a29b-f87ce64a3129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:01.937752 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd30a32-e1dc-48d8-a29b-f87ce64a3129 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:01.938372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd30a32-e1dc-48d8-a29b-f87ce64a3129 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:01.938779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd30a32-e1dc-48d8-a29b-f87ce64a3129 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:01.951936 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd30a32-e1dc-48d8-a29b-f87ce64a3129 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:01.953696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd30a32-e1dc-48d8-a29b-f87ce64a3129 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-fbd30a32-e1dc-48d8-a29b-f87ce64a3129 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:01.972441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:01.973520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:01.974424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:01.974424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:01.974424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:01.974424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:01.974424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:01.974424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:01.974424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:01.974424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:01.976882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 350/706] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:01.981533 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:01.987558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd30a32-e1dc-48d8-a29b-f87ce64a3129 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:02.008167 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822c49b-b6f7-43da-abdd-5953265c1971 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:02.046075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5fe5a-22f1-4eac-ab6e-a96052f91cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:02.046745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5fe5a-22f1-4eac-ab6e-a96052f91cc9 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:02.047030 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5fe5a-22f1-4eac-ab6e-a96052f91cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:02.047308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5fe5a-22f1-4eac-ab6e-a96052f91cc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:02.048155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5fe5a-22f1-4eac-ab6e-a96052f91cc9 None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=7d6be43abd85417c9fb7bbcb36a1f77d) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:02.048467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5fe5a-22f1-4eac-ab6e-a96052f91cc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:02.050911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5fe5a-22f1-4eac-ab6e-a96052f91cc9 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:02.057549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 357/707] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:01 2022] GET /identity/v3/role_assignments?user.id=7d6be43abd85417c9fb7bbcb36a1f77d&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:02.109851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd30a32-e1dc-48d8-a29b-f87ce64a3129 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:02.135828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:02.161716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6231b87-cb37-4124-93ea-9b7103b989c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:02.162795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6231b87-cb37-4124-93ea-9b7103b989c6 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:02.163257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6231b87-cb37-4124-93ea-9b7103b989c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:02.163698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6231b87-cb37-4124-93ea-9b7103b989c6 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:02.164708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6231b87-cb37-4124-93ea-9b7103b989c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:02.165169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6231b87-cb37-4124-93ea-9b7103b989c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:02.166807 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6231b87-cb37-4124-93ea-9b7103b989c6 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:02.186454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 351/708] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:02.192752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5fe5a-22f1-4eac-ab6e-a96052f91cc9 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:02.273928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f764a-0dc5-437d-bfdf-460cbef5b021 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:02.274585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f764a-0dc5-437d-bfdf-460cbef5b021 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:02.275061 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f764a-0dc5-437d-bfdf-460cbef5b021 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:02.276897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 352/709] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:02.282349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19506bbb-9065-407e-973c-ea0b12650a68 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:02.282867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19506bbb-9065-407e-973c-ea0b12650a68 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:02.283336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19506bbb-9065-407e-973c-ea0b12650a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:02.320131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-19506bbb-9065-407e-973c-ea0b12650a68 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:02.321517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5fe5a-22f1-4eac-ab6e-a96052f91cc9 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:02.321917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-19506bbb-9065-407e-973c-ea0b12650a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:02.364535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad9872-33d9-4133-9bb4-05fb1587b1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:02.365168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad9872-33d9-4133-9bb4-05fb1587b1ae None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:02.365454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad9872-33d9-4133-9bb4-05fb1587b1ae None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:02.365731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad9872-33d9-4133-9bb4-05fb1587b1ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:02.366579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad9872-33d9-4133-9bb4-05fb1587b1ae None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=8b476d6b82ae464b81fc8498dc0e68c7) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:02.366893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad9872-33d9-4133-9bb4-05fb1587b1ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:02.368322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad9872-33d9-4133-9bb4-05fb1587b1ae None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:02.375215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 358/710] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:02 2022] GET /identity/v3/role_assignments?user.id=8b476d6b82ae464b81fc8498dc0e68c7&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:02.377173 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f009b-9e46-4ac6-8138-bf80cc76afb5 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:02.377466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f009b-9e46-4ac6-8138-bf80cc76afb5 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:02.377757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f009b-9e46-4ac6-8138-bf80cc76afb5 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:02.379371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 359/711] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:02.385363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e88f90-eb1b-43a2-ba76-f19a9eb3cb52 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:02.385662 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e88f90-eb1b-43a2-ba76-f19a9eb3cb52 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:02.385931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e88f90-eb1b-43a2-ba76-f19a9eb3cb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:02.396119 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 353/712] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:33:02.406971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a0e88f90-eb1b-43a2-ba76-f19a9eb3cb52 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:02.408253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a0e88f90-eb1b-43a2-ba76-f19a9eb3cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:02.413000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:02.420258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19506bbb-9065-407e-973c-ea0b12650a68 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:02.511294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 360/713] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:33:02.536000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e88f90-eb1b-43a2-ba76-f19a9eb3cb52 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:02.551019 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19506bbb-9065-407e-973c-ea0b12650a68 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:02.566501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:02.596074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2cedf-0ce2-40ed-a56d-0422ad0836f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:02.597594 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2cedf-0ce2-40ed-a56d-0422ad0836f5 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:02.597894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2cedf-0ce2-40ed-a56d-0422ad0836f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:02.598177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2cedf-0ce2-40ed-a56d-0422ad0836f5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:02.598968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2cedf-0ce2-40ed-a56d-0422ad0836f5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:02.600362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2cedf-0ce2-40ed-a56d-0422ad0836f5 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-b1b2cedf-0ce2-40ed-a56d-0422ad0836f5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:02.616988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:02.617980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:02.618891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 354/714] 10.222.0.52 () {56 vars in 1129 bytes} [Mon Jul 25 08:33:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:02.626561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2cedf-0ce2-40ed-a56d-0422ad0836f5 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:02.664867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e88f90-eb1b-43a2-ba76-f19a9eb3cb52 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:02.714459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbbef0-eb92-45a1-8ae1-6447b50ea956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:02.715289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbbef0-eb92-45a1-8ae1-6447b50ea956 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:02.715574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbbef0-eb92-45a1-8ae1-6447b50ea956 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:02.715851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbbef0-eb92-45a1-8ae1-6447b50ea956 None admin] PATH_INFO: `/v3/services/network` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:02.716545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbbef0-eb92-45a1-8ae1-6447b50ea956 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:02.718098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbbef0-eb92-45a1-8ae1-6447b50ea956 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-25bbbef0-eb92-45a1-8ae1-6447b50ea956 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:02.746116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:02.747150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:02.747963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 361/715] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:02 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:02.754014 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbbef0-eb92-45a1-8ae1-6447b50ea956 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:02.757656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2cedf-0ce2-40ed-a56d-0422ad0836f5 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:02.760892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:02.793381 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6a9fd-aaad-4275-be4d-eb023244002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:02.794088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6a9fd-aaad-4275-be4d-eb023244002a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:02.794373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6a9fd-aaad-4275-be4d-eb023244002a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:02.794642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6a9fd-aaad-4275-be4d-eb023244002a None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:02.795359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6a9fd-aaad-4275-be4d-eb023244002a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:02.795660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6a9fd-aaad-4275-be4d-eb023244002a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:02.797055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6a9fd-aaad-4275-be4d-eb023244002a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:02.801908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 355/716] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Jul 25 08:33:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:02.807708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6a9fd-aaad-4275-be4d-eb023244002a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:02.903969 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbbef0-eb92-45a1-8ae1-6447b50ea956 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:02.938026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:02.943226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c6d19-7846-413f-9d18-1515dc9fe0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:02.943955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c6d19-7846-413f-9d18-1515dc9fe0ef None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:02.944248 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c6d19-7846-413f-9d18-1515dc9fe0ef None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:02.944716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c6d19-7846-413f-9d18-1515dc9fe0ef None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:02.945440 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c6d19-7846-413f-9d18-1515dc9fe0ef None admin] RBAC: Adding query filter params (name=network) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:02.945741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c6d19-7846-413f-9d18-1515dc9fe0ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:02.947099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c6d19-7846-413f-9d18-1515dc9fe0ef None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:02.952700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 362/717] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:02.958315 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c6d19-7846-413f-9d18-1515dc9fe0ef None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:03.063928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:03.087433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6a9fd-aaad-4275-be4d-eb023244002a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:03.104097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c6d19-7846-413f-9d18-1515dc9fe0ef None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:03.155215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6eeb6c-1751-4028-9946-ceee7ffc76ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:03.156684 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099007db-7cf4-42dd-85de-bb399d0f690c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:03.157589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6eeb6c-1751-4028-9946-ceee7ffc76ee None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:03.158492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099007db-7cf4-42dd-85de-bb399d0f690c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:03.158975 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6eeb6c-1751-4028-9946-ceee7ffc76ee None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:03.159447 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6eeb6c-1751-4028-9946-ceee7ffc76ee None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:03.159919 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099007db-7cf4-42dd-85de-bb399d0f690c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:03.160937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6eeb6c-1751-4028-9946-ceee7ffc76ee None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:03.161462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6eeb6c-1751-4028-9946-ceee7ffc76ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:03.163037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6eeb6c-1751-4028-9946-ceee7ffc76ee None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:03.164980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099007db-7cf4-42dd-85de-bb399d0f690c None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:03.165736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099007db-7cf4-42dd-85de-bb399d0f690c None admin] RBAC: Adding query filter params (type=network) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:03.166052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099007db-7cf4-42dd-85de-bb399d0f690c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:03.167489 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099007db-7cf4-42dd-85de-bb399d0f690c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:03.170746 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 356/718] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Jul 25 08:33:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:03.181565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 363/719] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:04.230139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b58a91-0a74-4d0b-b274-b8f1bdcd1c43 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:04.230661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b58a91-0a74-4d0b-b274-b8f1bdcd1c43 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:04.230775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b58a91-0a74-4d0b-b274-b8f1bdcd1c43 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:04.232351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 357/720] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:04.237676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b95e5c-1207-4e4e-bf57-492d67115ac6 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:04.238092 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b95e5c-1207-4e4e-bf57-492d67115ac6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:04.238262 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b95e5c-1207-4e4e-bf57-492d67115ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:04.265583 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-81b95e5c-1207-4e4e-bf57-492d67115ac6 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:04.266787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-81b95e5c-1207-4e4e-bf57-492d67115ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:04.371018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 364/721] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:33:04.409953 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b58a91-0a74-4d0b-b274-b8f1bdcd1c43 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:04.570814 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b58a91-0a74-4d0b-b274-b8f1bdcd1c43 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:04.578611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:04.612543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce74219-445d-4e22-9cd6-79fddcc6405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:04.613442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce74219-445d-4e22-9cd6-79fddcc6405f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:04.613724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce74219-445d-4e22-9cd6-79fddcc6405f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:04.613998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce74219-445d-4e22-9cd6-79fddcc6405f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:04.631487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce74219-445d-4e22-9cd6-79fddcc6405f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-bce74219-445d-4e22-9cd6-79fddcc6405f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:04.639416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:04.640419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:04.641351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 358/722] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:33:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:04.647499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b95e5c-1207-4e4e-bf57-492d67115ac6 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:04.714377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:04.758921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b95e5c-1207-4e4e-bf57-492d67115ac6 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:04.794035 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a2b68-5856-4f22-8239-1b16ec9a989a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:04.794717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a2b68-5856-4f22-8239-1b16ec9a989a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:04.795008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a2b68-5856-4f22-8239-1b16ec9a989a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:04.795291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a2b68-5856-4f22-8239-1b16ec9a989a None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:04.796044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a2b68-5856-4f22-8239-1b16ec9a989a None admin] RBAC: Adding query filter params (name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:04.796356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a2b68-5856-4f22-8239-1b16ec9a989a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:04.797772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a2b68-5856-4f22-8239-1b16ec9a989a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:04.807261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 365/723] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:33:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:04.815133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce74219-445d-4e22-9cd6-79fddcc6405f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:04.883777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc50284-b4c3-4fc2-9988-6a2736a4372d None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:04.883970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc50284-b4c3-4fc2-9988-6a2736a4372d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:04.884258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc50284-b4c3-4fc2-9988-6a2736a4372d None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:04.886861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 366/724] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:04.890622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f617e-1a6a-4721-885c-aac75a076260 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:04.890918 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f617e-1a6a-4721-885c-aac75a076260 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:04.891205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f617e-1a6a-4721-885c-aac75a076260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:04.927193 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a13f617e-1a6a-4721-885c-aac75a076260 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:04.928364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a13f617e-1a6a-4721-885c-aac75a076260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:04.937408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce74219-445d-4e22-9cd6-79fddcc6405f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:04.990625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c178e-afc3-4ba8-b31f-301c492381ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:04.991439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c178e-afc3-4ba8-b31f-301c492381ef None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:04.991724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c178e-afc3-4ba8-b31f-301c492381ef None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:04.992006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c178e-afc3-4ba8-b31f-301c492381ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.017080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c178e-afc3-4ba8-b31f-301c492381ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:05.018546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c178e-afc3-4ba8-b31f-301c492381ef None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:05.048674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 367/725] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 08:33:05.048674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d87ba-4dba-485f-a92c-d93aca1ee524 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.048674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d87ba-4dba-485f-a92c-d93aca1ee524 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.048674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d87ba-4dba-485f-a92c-d93aca1ee524 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.048674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 368/726] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-950c178e-afc3-4ba8-b31f-301c492381ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:05.054678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:05.055555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:05.056441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:05.056441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:05.056441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:05.056441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:05.056688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284497e-53c1-4a9c-9fef-1828c4a6a4da None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.056688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284497e-53c1-4a9c-9fef-1828c4a6a4da None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.056688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284497e-53c1-4a9c-9fef-1828c4a6a4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.058876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 359/727] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:05.064679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c178e-afc3-4ba8-b31f-301c492381ef None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:05.083818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d284497e-53c1-4a9c-9fef-1828c4a6a4da None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:05.085057 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d284497e-53c1-4a9c-9fef-1828c4a6a4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:05.088807 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:05.139527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:05.163693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 369/728] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:33:05.164663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284497e-53c1-4a9c-9fef-1828c4a6a4da None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:05.183671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c178e-afc3-4ba8-b31f-301c492381ef None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:05.221058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a40aa-e487-4893-8068-a49437ce7102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:05.222202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a40aa-e487-4893-8068-a49437ce7102 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.222684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a40aa-e487-4893-8068-a49437ce7102 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.223139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a40aa-e487-4893-8068-a49437ce7102 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.224659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a40aa-e487-4893-8068-a49437ce7102 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:05.225309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a40aa-e487-4893-8068-a49437ce7102 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:05.226950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a40aa-e487-4893-8068-a49437ce7102 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:05.241281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 360/729] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:05.247264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a40aa-e487-4893-8068-a49437ce7102 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:05.277028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284497e-53c1-4a9c-9fef-1828c4a6a4da None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:05.314589 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d59a69-c6ea-4bc2-8a72-a97190ef0dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:05.315499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d59a69-c6ea-4bc2-8a72-a97190ef0dc8 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.315830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d59a69-c6ea-4bc2-8a72-a97190ef0dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.316110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d59a69-c6ea-4bc2-8a72-a97190ef0dc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.327784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d59a69-c6ea-4bc2-8a72-a97190ef0dc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-a5d59a69-c6ea-4bc2-8a72-a97190ef0dc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:05.336166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:05.337175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:05.338007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:05.338007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:05.338007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:05.338007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:05.338007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:05.338007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:05.338007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:05.338007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:05.338007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:05.338007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:05.338007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:05.338007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:05.340815 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 370/730] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:33:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:05.341831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d59a69-c6ea-4bc2-8a72-a97190ef0dc8 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:05.348691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:05.374032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a40aa-e487-4893-8068-a49437ce7102 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:05.391478 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:05.415291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327ab04-a66a-40a3-a2f2-7edc2d54397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:05.416196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327ab04-a66a-40a3-a2f2-7edc2d54397f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.416552 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327ab04-a66a-40a3-a2f2-7edc2d54397f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.416930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327ab04-a66a-40a3-a2f2-7edc2d54397f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.428124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327ab04-a66a-40a3-a2f2-7edc2d54397f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-6327ab04-a66a-40a3-a2f2-7edc2d54397f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:05.436686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:05.437620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:05.438599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 361/731] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:33:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:05.439484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327ab04-a66a-40a3-a2f2-7edc2d54397f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:05.459441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d59a69-c6ea-4bc2-8a72-a97190ef0dc8 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:05.496883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617ac47-09c2-4470-b703-029038b0023c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:05.497983 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617ac47-09c2-4470-b703-029038b0023c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.498499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617ac47-09c2-4470-b703-029038b0023c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.499006 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617ac47-09c2-4470-b703-029038b0023c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.507670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617ac47-09c2-4470-b703-029038b0023c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:05.509449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617ac47-09c2-4470-b703-029038b0023c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-5617ac47-09c2-4470-b703-029038b0023c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:05.525705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:05.526681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:05.527868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:05.527868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:05.527868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:05.527868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:05.527868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:05.527868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:05.527868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:05.527868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:05.527868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 08:33:05.527868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:05.530614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 371/732] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Jul 25 08:33:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:05.531678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617ac47-09c2-4470-b703-029038b0023c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:05.554976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327ab04-a66a-40a3-a2f2-7edc2d54397f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:05.592740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576380a3-de5d-4264-b167-bbb9e9c7e932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:05.593413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576380a3-de5d-4264-b167-bbb9e9c7e932 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.593698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576380a3-de5d-4264-b167-bbb9e9c7e932 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.593974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576380a3-de5d-4264-b167-bbb9e9c7e932 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.594723 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576380a3-de5d-4264-b167-bbb9e9c7e932 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:05.595028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576380a3-de5d-4264-b167-bbb9e9c7e932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:05.596397 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576380a3-de5d-4264-b167-bbb9e9c7e932 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:05.602361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:05.610787 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 362/733] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:33:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:05.611793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576380a3-de5d-4264-b167-bbb9e9c7e932 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:05.645397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617ac47-09c2-4470-b703-029038b0023c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:05.647486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:05.682373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adb846-6b25-430a-bdc5-98955f21cb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:05.683046 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adb846-6b25-430a-bdc5-98955f21cb01 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.683329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adb846-6b25-430a-bdc5-98955f21cb01 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.683602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adb846-6b25-430a-bdc5-98955f21cb01 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.684344 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9adb846-6b25-430a-bdc5-98955f21cb01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:05.684666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9adb846-6b25-430a-bdc5-98955f21cb01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:05.686024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9adb846-6b25-430a-bdc5-98955f21cb01 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:05.694636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 372/734] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:33:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:05.696075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e116c5-d841-41b6-8db8-b95a7bf047a8 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.696518 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e116c5-d841-41b6-8db8-b95a7bf047a8 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.696847 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e116c5-d841-41b6-8db8-b95a7bf047a8 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.698193 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 373/735] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:05.699606 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e116c5-d841-41b6-8db8-b95a7bf047a8 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:05.717536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576380a3-de5d-4264-b167-bbb9e9c7e932 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:05.751001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596edc4a-07ad-4abd-9e62-e859f52bf8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:05.751674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596edc4a-07ad-4abd-9e62-e859f52bf8b4 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.751957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596edc4a-07ad-4abd-9e62-e859f52bf8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.752235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596edc4a-07ad-4abd-9e62-e859f52bf8b4 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.753640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596edc4a-07ad-4abd-9e62-e859f52bf8b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:05.753947 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596edc4a-07ad-4abd-9e62-e859f52bf8b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:05.755371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596edc4a-07ad-4abd-9e62-e859f52bf8b4 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:05.772461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 363/736] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Jul 25 08:33:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:05.773741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268399b-edaf-456f-8f27-8ca5662f1600 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.774035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268399b-edaf-456f-8f27-8ca5662f1600 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.774315 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268399b-edaf-456f-8f27-8ca5662f1600 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.779514 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 364/737] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:05.779514 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268399b-edaf-456f-8f27-8ca5662f1600 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:05.805714 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e116c5-d841-41b6-8db8-b95a7bf047a8 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:05.835556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:05.841974 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc85cd-f862-4285-873c-99f867ab9d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:05.842775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc85cd-f862-4285-873c-99f867ab9d85 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.843063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc85cd-f862-4285-873c-99f867ab9d85 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.843343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc85cd-f862-4285-873c-99f867ab9d85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.853504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc85cd-f862-4285-873c-99f867ab9d85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:05.854958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc85cd-f862-4285-873c-99f867ab9d85 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-fbdc85cd-f862-4285-873c-99f867ab9d85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:05.872850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:05.873719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:05.874595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:05.874595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:05.874595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:05.874595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:05.874595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 374/738] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:05.875225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175ebac-63df-4cad-ba63-f01d63f17142 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.875501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175ebac-63df-4cad-ba63-f01d63f17142 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.875762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175ebac-63df-4cad-ba63-f01d63f17142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.880843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268399b-edaf-456f-8f27-8ca5662f1600 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:05.893188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4175ebac-63df-4cad-ba63-f01d63f17142 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:05.894951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4175ebac-63df-4cad-ba63-f01d63f17142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:05.905839 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:05.915641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61158e7-3b25-4863-a987-b112487f3291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:05.916437 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61158e7-3b25-4863-a987-b112487f3291 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.916753 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61158e7-3b25-4863-a987-b112487f3291 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.917034 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61158e7-3b25-4863-a987-b112487f3291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.926966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61158e7-3b25-4863-a987-b112487f3291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:05.928410 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61158e7-3b25-4863-a987-b112487f3291 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-a61158e7-3b25-4863-a987-b112487f3291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:05.947167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:05.948125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:05.949053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:05.949053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:05.949053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:05.949053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:05.949053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 365/739] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:05.949323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61158e7-3b25-4863-a987-b112487f3291 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:05.965765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 375/740] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:33:05.967030 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e4b0a-805d-4db1-a3ae-33c07bcf2c26 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:05.967315 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e4b0a-805d-4db1-a3ae-33c07bcf2c26 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:05.967588 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e4b0a-805d-4db1-a3ae-33c07bcf2c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:05.992497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-748e4b0a-805d-4db1-a3ae-33c07bcf2c26 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:05.993729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-748e4b0a-805d-4db1-a3ae-33c07bcf2c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:06.054379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61158e7-3b25-4863-a987-b112487f3291 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:06.059547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 376/741] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 08:33:06.060370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e4b0a-805d-4db1-a3ae-33c07bcf2c26 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:06.071800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:06.096459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc4f72-0a67-487c-9866-ac89a940beb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:06.097185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc4f72-0a67-487c-9866-ac89a940beb9 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:06.097475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc4f72-0a67-487c-9866-ac89a940beb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:06.097756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc4f72-0a67-487c-9866-ac89a940beb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:06.138570 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:06.162358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e4b0a-805d-4db1-a3ae-33c07bcf2c26 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:06.163920 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc4f72-0a67-487c-9866-ac89a940beb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:06.165389 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc4f72-0a67-487c-9866-ac89a940beb9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:06.197126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9db71-494c-4a0c-91e9-04208d1c8d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:06.197801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9db71-494c-4a0c-91e9-04208d1c8d38 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:06.198095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9db71-494c-4a0c-91e9-04208d1c8d38 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:06.198376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9db71-494c-4a0c-91e9-04208d1c8d38 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:06.199106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9db71-494c-4a0c-91e9-04208d1c8d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:06.199415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9db71-494c-4a0c-91e9-04208d1c8d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:06.200817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9db71-494c-4a0c-91e9-04208d1c8d38 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:06.212078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 377/742] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:06.212893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9db71-494c-4a0c-91e9-04208d1c8d38 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:06.266327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 366/743] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 08:33:06.267138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc4f72-0a67-487c-9866-ac89a940beb9 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:06.279045 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:06.313702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9db71-494c-4a0c-91e9-04208d1c8d38 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:06.347210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146d90a-0ae6-43ec-8afb-22bff96e0494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:06.347873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146d90a-0ae6-43ec-8afb-22bff96e0494 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:06.348153 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146d90a-0ae6-43ec-8afb-22bff96e0494 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:06.348425 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146d90a-0ae6-43ec-8afb-22bff96e0494 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:06.349178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146d90a-0ae6-43ec-8afb-22bff96e0494 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:06.349488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146d90a-0ae6-43ec-8afb-22bff96e0494 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:06.350849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146d90a-0ae6-43ec-8afb-22bff96e0494 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:06.353106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:06.363513 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 378/744] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:06.364193 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146d90a-0ae6-43ec-8afb-22bff96e0494 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:06.369526 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc4f72-0a67-487c-9866-ac89a940beb9 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:06.405595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e060b-16e8-4209-a15b-e8bf9cdd796d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:06.406259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e060b-16e8-4209-a15b-e8bf9cdd796d None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:06.406544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e060b-16e8-4209-a15b-e8bf9cdd796d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:06.406825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e060b-16e8-4209-a15b-e8bf9cdd796d None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:06.407563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e060b-16e8-4209-a15b-e8bf9cdd796d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:06.408947 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e060b-16e8-4209-a15b-e8bf9cdd796d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:06.424226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 367/745] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:33:06 2022] POST /identity/v3/services => generated 251 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:06.425806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e060b-16e8-4209-a15b-e8bf9cdd796d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:06.469534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146d90a-0ae6-43ec-8afb-22bff96e0494 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:06.523154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76233b9a-7d06-4bb8-9779-f1ae0bc2256c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:06.523843 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76233b9a-7d06-4bb8-9779-f1ae0bc2256c None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:06.524128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76233b9a-7d06-4bb8-9779-f1ae0bc2256c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:06.524403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76233b9a-7d06-4bb8-9779-f1ae0bc2256c None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:06.525227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76233b9a-7d06-4bb8-9779-f1ae0bc2256c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:06.526652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76233b9a-7d06-4bb8-9779-f1ae0bc2256c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:06.530203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:06.544946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 379/746] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:33:06 2022] POST /identity/v3/services => generated 240 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:06.545829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76233b9a-7d06-4bb8-9779-f1ae0bc2256c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:06.595598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e060b-16e8-4209-a15b-e8bf9cdd796d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:06.630667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:06.656886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108671ca-43af-4577-a653-52bdb757d907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:06.658057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108671ca-43af-4577-a653-52bdb757d907 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:06.658551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108671ca-43af-4577-a653-52bdb757d907 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:06.659010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108671ca-43af-4577-a653-52bdb757d907 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:06.670756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108671ca-43af-4577-a653-52bdb757d907 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:06.673058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108671ca-43af-4577-a653-52bdb757d907 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-108671ca-43af-4577-a653-52bdb757d907 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:06.691286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:06.692297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:06.693291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:06.693291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:06.693291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:06.693291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:06.693291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:06.693291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:06.693291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:06.693291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:06.693291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:33:06.693291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:06.693866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76233b9a-7d06-4bb8-9779-f1ae0bc2256c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:06.693943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 368/747] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Jul 25 08:33:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:06.693943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108671ca-43af-4577-a653-52bdb757d907 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:06.740505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07c505-29dc-48b0-9e04-363733a1f7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:06.741357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07c505-29dc-48b0-9e04-363733a1f7be None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:06.741643 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07c505-29dc-48b0-9e04-363733a1f7be None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:06.741926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07c505-29dc-48b0-9e04-363733a1f7be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:06.753865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07c505-29dc-48b0-9e04-363733a1f7be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:06.755328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07c505-29dc-48b0-9e04-363733a1f7be None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-6f07c505-29dc-48b0-9e04-363733a1f7be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:06.776349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:06.777368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:06.778811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:06.778811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:06.778811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:06.778811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:06.778811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 380/748] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:06.778811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07c505-29dc-48b0-9e04-363733a1f7be None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:06.823185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108671ca-43af-4577-a653-52bdb757d907 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:06.852661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:06.863353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e2caf-1ba6-4319-b55c-d504f9b3747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:06.863647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e2caf-1ba6-4319-b55c-d504f9b3747f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:06.863997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e2caf-1ba6-4319-b55c-d504f9b3747f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:06.864346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e2caf-1ba6-4319-b55c-d504f9b3747f None admin] PATH_INFO: `/v3/users/swift` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:06.872675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e2caf-1ba6-4319-b55c-d504f9b3747f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:06.874212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e2caf-1ba6-4319-b55c-d504f9b3747f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-457e2caf-1ba6-4319-b55c-d504f9b3747f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:06.889716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:06.890843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:06.891992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:06.891992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:06.891992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:06.891992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:06.891992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:06.891992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:06.891992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:06.891992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:06.891992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 08:33:06.891992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:06.893692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 369/749] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:33:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:06.894872 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e2caf-1ba6-4319-b55c-d504f9b3747f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:06.914838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07c505-29dc-48b0-9e04-363733a1f7be None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:06.917312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:06.966168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b47ddb-08e6-4a7a-91c8-95f29bbb9055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:06.966794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b47ddb-08e6-4a7a-91c8-95f29bbb9055 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:06.967634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b47ddb-08e6-4a7a-91c8-95f29bbb9055 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:06.967915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b47ddb-08e6-4a7a-91c8-95f29bbb9055 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:06.969128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b47ddb-08e6-4a7a-91c8-95f29bbb9055 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:06.969448 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b47ddb-08e6-4a7a-91c8-95f29bbb9055 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:06.972111 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b47ddb-08e6-4a7a-91c8-95f29bbb9055 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:06.996660 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 381/750] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Jul 25 08:33:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:06.997634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b47ddb-08e6-4a7a-91c8-95f29bbb9055 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:07.009999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e2caf-1ba6-4319-b55c-d504f9b3747f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:07.055625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c6de9-9101-44f1-807d-0220ec4c82a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:07.056306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c6de9-9101-44f1-807d-0220ec4c82a6 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:07.056613 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c6de9-9101-44f1-807d-0220ec4c82a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:07.057340 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c6de9-9101-44f1-807d-0220ec4c82a6 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:07.058088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c6de9-9101-44f1-807d-0220ec4c82a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:07.058394 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c6de9-9101-44f1-807d-0220ec4c82a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:07.060511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c6de9-9101-44f1-807d-0220ec4c82a6 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:07.078501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 370/751] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:07.079471 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c6de9-9101-44f1-807d-0220ec4c82a6 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:07.122748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b47ddb-08e6-4a7a-91c8-95f29bbb9055 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:07.130622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:07.158558 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d5e1c-a0f4-408d-aa4a-8d67a3b38286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:07.159293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d5e1c-a0f4-408d-aa4a-8d67a3b38286 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:07.159580 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d5e1c-a0f4-408d-aa4a-8d67a3b38286 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:07.159859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d5e1c-a0f4-408d-aa4a-8d67a3b38286 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:07.160732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d5e1c-a0f4-408d-aa4a-8d67a3b38286 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:07.161041 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d5e1c-a0f4-408d-aa4a-8d67a3b38286 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:07.162534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d5e1c-a0f4-408d-aa4a-8d67a3b38286 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:07.180865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 382/752] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Jul 25 08:33:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:07.181702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d5e1c-a0f4-408d-aa4a-8d67a3b38286 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:07.191476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c6de9-9101-44f1-807d-0220ec4c82a6 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:07.203595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:07.231763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392992c4-f87a-4875-98c8-fb7b53059655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:07.232815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392992c4-f87a-4875-98c8-fb7b53059655 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:07.233343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392992c4-f87a-4875-98c8-fb7b53059655 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:07.233788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392992c4-f87a-4875-98c8-fb7b53059655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:07.245237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392992c4-f87a-4875-98c8-fb7b53059655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:07.247156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392992c4-f87a-4875-98c8-fb7b53059655 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-392992c4-f87a-4875-98c8-fb7b53059655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:07.267011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:07.268627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:07.270008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:07.270008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:07.270008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:07.270008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:07.272013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 371/753] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:07.273080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392992c4-f87a-4875-98c8-fb7b53059655 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:07.291145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d5e1c-a0f4-408d-aa4a-8d67a3b38286 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:07.328114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395f69a-e94f-40d4-a589-a90835536e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:07.328951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395f69a-e94f-40d4-a589-a90835536e2a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:07.329246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395f69a-e94f-40d4-a589-a90835536e2a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:07.329728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395f69a-e94f-40d4-a589-a90835536e2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:07.337158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395f69a-e94f-40d4-a589-a90835536e2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:07.338589 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395f69a-e94f-40d4-a589-a90835536e2a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-c395f69a-e94f-40d4-a589-a90835536e2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:07.352515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:07.353567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:07.354691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:07.354691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:07.354691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:07.354691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:07.354691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:07.354691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:07.354691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:07.354691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:07.354691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 383/754] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:07.355312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395f69a-e94f-40d4-a589-a90835536e2a None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:07.394970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:07.399442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392992c4-f87a-4875-98c8-fb7b53059655 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:07.436517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7c4e5-77f0-4c9e-bd07-0fadc32dd75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:07.437491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7c4e5-77f0-4c9e-bd07-0fadc32dd75c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:07.437962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7c4e5-77f0-4c9e-bd07-0fadc32dd75c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:07.438653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:07.438909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7c4e5-77f0-4c9e-bd07-0fadc32dd75c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:07.459574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395f69a-e94f-40d4-a589-a90835536e2a None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:07.493129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2531827-31a9-43e3-b189-690bb692a6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:07.493720 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2531827-31a9-43e3-b189-690bb692a6ee None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:07.494000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2531827-31a9-43e3-b189-690bb692a6ee None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:07.494273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2531827-31a9-43e3-b189-690bb692a6ee None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:07.494996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2531827-31a9-43e3-b189-690bb692a6ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:07.495296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2531827-31a9-43e3-b189-690bb692a6ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:07.496689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2531827-31a9-43e3-b189-690bb692a6ee None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:07.509141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 384/755] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:07.509998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2531827-31a9-43e3-b189-690bb692a6ee None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:07.517014 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7c4e5-77f0-4c9e-bd07-0fadc32dd75c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:07.519394 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7c4e5-77f0-4c9e-bd07-0fadc32dd75c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:07.614702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2531827-31a9-43e3-b189-690bb692a6ee None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:07.622368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:07.629417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 372/756] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:33:07.630882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7c4e5-77f0-4c9e-bd07-0fadc32dd75c None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:07.653367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7653004-3c10-456d-ac8c-1b8e83c51492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:07.654034 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7653004-3c10-456d-ac8c-1b8e83c51492 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:07.654321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7653004-3c10-456d-ac8c-1b8e83c51492 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:07.654596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7653004-3c10-456d-ac8c-1b8e83c51492 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:07.655387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7653004-3c10-456d-ac8c-1b8e83c51492 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:07.655691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7653004-3c10-456d-ac8c-1b8e83c51492 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:07.657722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7653004-3c10-456d-ac8c-1b8e83c51492 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:07.666255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:07.669949 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 385/757] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:07.670987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7653004-3c10-456d-ac8c-1b8e83c51492 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:07.761099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7c4e5-77f0-4c9e-bd07-0fadc32dd75c None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:07.776221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7653004-3c10-456d-ac8c-1b8e83c51492 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:07.799376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497a643-7f5b-46a2-8898-e3e11c7e4f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:07.800833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497a643-7f5b-46a2-8898-e3e11c7e4f15 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:07.801115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497a643-7f5b-46a2-8898-e3e11c7e4f15 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:07.801490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497a643-7f5b-46a2-8898-e3e11c7e4f15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:07.809183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497a643-7f5b-46a2-8898-e3e11c7e4f15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:07.810623 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497a643-7f5b-46a2-8898-e3e11c7e4f15 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:07.812187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6ca8a-6df7-40c3-8e64-1906a7e4199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:07.813022 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6ca8a-6df7-40c3-8e64-1906a7e4199f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:07.813306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6ca8a-6df7-40c3-8e64-1906a7e4199f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:07.813585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6ca8a-6df7-40c3-8e64-1906a7e4199f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:07.825118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6ca8a-6df7-40c3-8e64-1906a7e4199f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-9497a643-7f5b-46a2-8898-e3e11c7e4f15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:07.825694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:07.826531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:07.827789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:07.827789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:07.827789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:07.827789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:07.827789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:07.827789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:07.827789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:07.827789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:07.828423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6ca8a-6df7-40c3-8e64-1906a7e4199f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:07.828494 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 373/758] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:07.828494 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497a643-7f5b-46a2-8898-e3e11c7e4f15 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-52d6ca8a-6df7-40c3-8e64-1906a7e4199f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:07.848831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:07.850921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:07.853136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:07.853136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:07.853136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:07.853136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:07.853136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 386/759] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:07.853136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6ca8a-6df7-40c3-8e64-1906a7e4199f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:07.896218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:07.904966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:07.946342 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497a643-7f5b-46a2-8898-e3e11c7e4f15 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:07.953267 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6ca8a-6df7-40c3-8e64-1906a7e4199f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:07.983164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6bfa3-bb5c-4cb8-b334-97373739eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:07.984082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6bfa3-bb5c-4cb8-b334-97373739eceb None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:07.984537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6bfa3-bb5c-4cb8-b334-97373739eceb None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:07.985050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6bfa3-bb5c-4cb8-b334-97373739eceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:07.986214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6bfa3-bb5c-4cb8-b334-97373739eceb None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=c151c53202f24faeb3accf802716652f) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:07.986675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6bfa3-bb5c-4cb8-b334-97373739eceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:07.988179 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f7380-0143-4255-b585-8cd38d633363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:07.988865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f7380-0143-4255-b585-8cd38d633363 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:07.989147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f7380-0143-4255-b585-8cd38d633363 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:07.989427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f7380-0143-4255-b585-8cd38d633363 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:07.989537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6bfa3-bb5c-4cb8-b334-97373739eceb None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:07.990216 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f7380-0143-4255-b585-8cd38d633363 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:07.990523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f7380-0143-4255-b585-8cd38d633363 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:07.991930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f7380-0143-4255-b585-8cd38d633363 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:07.996509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 374/760] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:07 2022] GET /identity/v3/role_assignments?user.id=c151c53202f24faeb3accf802716652f&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:07.997547 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6bfa3-bb5c-4cb8-b334-97373739eceb None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:08.005870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 387/761] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:08.011891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f7380-0143-4255-b585-8cd38d633363 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:08.116334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6bfa3-bb5c-4cb8-b334-97373739eceb None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:08.120055 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f7380-0143-4255-b585-8cd38d633363 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:08.134657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:08.143919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:08.159633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a87a77-901b-47db-ab77-9364e29e320b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:08.160588 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a87a77-901b-47db-ab77-9364e29e320b None admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:08.160943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a87a77-901b-47db-ab77-9364e29e320b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:08.161218 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a87a77-901b-47db-ab77-9364e29e320b None admin] PATH_INFO: `/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/8b476d6b82ae464b81fc8498dc0e68c7/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:08.161707 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14227523-c91b-4fee-8fe1-2ec92b1800d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:08.162764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14227523-c91b-4fee-8fe1-2ec92b1800d3 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:08.163205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14227523-c91b-4fee-8fe1-2ec92b1800d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:08.163318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a87a77-901b-47db-ab77-9364e29e320b None admin] RBAC: Authorizing `identity:create_grant(project_id=2ee27c4da77c47428e02fd65e890d256, user_id=8b476d6b82ae464b81fc8498dc0e68c7, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:08.163718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14227523-c91b-4fee-8fe1-2ec92b1800d3 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:08.164633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14227523-c91b-4fee-8fe1-2ec92b1800d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:08.164762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a87a77-901b-47db-ab77-9364e29e320b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:08.165197 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14227523-c91b-4fee-8fe1-2ec92b1800d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:08.166713 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14227523-c91b-4fee-8fe1-2ec92b1800d3 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:08.181879 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 375/762] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:08.187822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14227523-c91b-4fee-8fe1-2ec92b1800d3 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:08.209176 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 388/763] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:33:08 2022] PUT /identity/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/8b476d6b82ae464b81fc8498dc0e68c7/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:33:08.309123 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14227523-c91b-4fee-8fe1-2ec92b1800d3 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:08.349602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6acb3-c9cf-4e18-a6da-63428a170605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:08.350425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6acb3-c9cf-4e18-a6da-63428a170605 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:08.350714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6acb3-c9cf-4e18-a6da-63428a170605 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:08.350995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6acb3-c9cf-4e18-a6da-63428a170605 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:08.359077 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6acb3-c9cf-4e18-a6da-63428a170605 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:08.360588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6acb3-c9cf-4e18-a6da-63428a170605 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-f1b6acb3-c9cf-4e18-a6da-63428a170605 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:08.376481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:08.377475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:08.378897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:08.378897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:08.378897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:08.378897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:08.378897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:08.378897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:08.378897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:08.378897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:08.378897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 376/764] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:08.384422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a87a77-901b-47db-ab77-9364e29e320b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:08.496274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a87a77-901b-47db-ab77-9364e29e320b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:08.534937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97bc24-d8f9-4ef1-bdcf-a218c9e6668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:08.535940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97bc24-d8f9-4ef1-bdcf-a218c9e6668e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:08.536410 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97bc24-d8f9-4ef1-bdcf-a218c9e6668e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:08.536922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97bc24-d8f9-4ef1-bdcf-a218c9e6668e None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:08.537953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c97bc24-d8f9-4ef1-bdcf-a218c9e6668e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:08.538446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c97bc24-d8f9-4ef1-bdcf-a218c9e6668e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:08.540052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c97bc24-d8f9-4ef1-bdcf-a218c9e6668e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:08.553238 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 389/765] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:08.559464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6acb3-c9cf-4e18-a6da-63428a170605 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:08.607742 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:08.615747 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d6f3c-49fb-4b47-82c2-626439592974 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:08.616147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d6f3c-49fb-4b47-82c2-626439592974 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:08.616506 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d6f3c-49fb-4b47-82c2-626439592974 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:08.619270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 390/766] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:08.623422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59968ac-5834-43f6-b172-bc5715b9b673 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:08.623776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59968ac-5834-43f6-b172-bc5715b9b673 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:08.624130 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59968ac-5834-43f6-b172-bc5715b9b673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:08.644012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c59968ac-5834-43f6-b172-bc5715b9b673 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:08.644910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c59968ac-5834-43f6-b172-bc5715b9b673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:08.673410 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:08.688352 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6acb3-c9cf-4e18-a6da-63428a170605 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:08.724725 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 391/767] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:08.733308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdeda79-9e00-45fa-9182-a2f50ddf4d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:08.734201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdeda79-9e00-45fa-9182-a2f50ddf4d87 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:08.734698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdeda79-9e00-45fa-9182-a2f50ddf4d87 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:08.735159 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdeda79-9e00-45fa-9182-a2f50ddf4d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:08.736186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdeda79-9e00-45fa-9182-a2f50ddf4d87 None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=7d6be43abd85417c9fb7bbcb36a1f77d) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:08.736731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdeda79-9e00-45fa-9182-a2f50ddf4d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:08.738382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdeda79-9e00-45fa-9182-a2f50ddf4d87 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:08.744697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 377/768] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:08 2022] GET /identity/v3/role_assignments?user.id=7d6be43abd85417c9fb7bbcb36a1f77d&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:08.748634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59968ac-5834-43f6-b172-bc5715b9b673 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:08.805663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c5489-cb05-419b-a96f-a6a1145e0e82 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:08.806013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c5489-cb05-419b-a96f-a6a1145e0e82 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:08.806236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c5489-cb05-419b-a96f-a6a1145e0e82 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:08.807898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 378/769] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:08.813363 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1abe6c-767c-4496-b01d-d4b5ee30191b None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:08.813629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1abe6c-767c-4496-b01d-d4b5ee30191b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:08.813854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1abe6c-767c-4496-b01d-d4b5ee30191b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:08.833608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-df1abe6c-767c-4496-b01d-d4b5ee30191b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:08.834845 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-df1abe6c-767c-4496-b01d-d4b5ee30191b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:08.920419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59968ac-5834-43f6-b172-bc5715b9b673 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:08.935323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 379/770] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:08.961618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1abe6c-767c-4496-b01d-d4b5ee30191b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:08.997793 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f6aac-1276-4194-8a27-99131e191622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:08.998646 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f6aac-1276-4194-8a27-99131e191622 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:08.998930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f6aac-1276-4194-8a27-99131e191622 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:08.999213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f6aac-1276-4194-8a27-99131e191622 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:08.999890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f6aac-1276-4194-8a27-99131e191622 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:09.001335 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f6aac-1276-4194-8a27-99131e191622 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:09.016539 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-178f6aac-1276-4194-8a27-99131e191622 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:09.020545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:09.021710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:09.022581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 392/771] 10.222.0.52 () {56 vars in 1129 bytes} [Mon Jul 25 08:33:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:09.030320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f6aac-1276-4194-8a27-99131e191622 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:09.074762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:09.106052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1abe6c-767c-4496-b01d-d4b5ee30191b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:09.149049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60510b-28a3-413b-a446-c1092336421f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:09.149860 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60510b-28a3-413b-a446-c1092336421f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:09.150145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60510b-28a3-413b-a446-c1092336421f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:09.150422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60510b-28a3-413b-a446-c1092336421f None admin] PATH_INFO: `/v3/services/network` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:09.151310 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60510b-28a3-413b-a446-c1092336421f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:09.154375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60510b-28a3-413b-a446-c1092336421f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-bf60510b-28a3-413b-a446-c1092336421f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:09.173336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:09.176185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:09.178222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 380/772] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:08 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:09.182577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60510b-28a3-413b-a446-c1092336421f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:09.211054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f6aac-1276-4194-8a27-99131e191622 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:09.250284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2869eb4-5b17-4fe7-b1fc-9e09d1389e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:09.251263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2869eb4-5b17-4fe7-b1fc-9e09d1389e60 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:09.251555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2869eb4-5b17-4fe7-b1fc-9e09d1389e60 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:09.251837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2869eb4-5b17-4fe7-b1fc-9e09d1389e60 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:09.252645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2869eb4-5b17-4fe7-b1fc-9e09d1389e60 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:09.253619 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2869eb4-5b17-4fe7-b1fc-9e09d1389e60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:09.255006 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2869eb4-5b17-4fe7-b1fc-9e09d1389e60 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:09.260680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 393/773] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Jul 25 08:33:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:09.266293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2869eb4-5b17-4fe7-b1fc-9e09d1389e60 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:09.370137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:09.373481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60510b-28a3-413b-a446-c1092336421f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:09.410389 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a22e00-6cb2-4807-a5e8-a049d950233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:09.411167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a22e00-6cb2-4807-a5e8-a049d950233d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:09.411463 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a22e00-6cb2-4807-a5e8-a049d950233d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:09.411746 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a22e00-6cb2-4807-a5e8-a049d950233d None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:09.412471 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a22e00-6cb2-4807-a5e8-a049d950233d None admin] RBAC: Adding query filter params (name=network) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:09.412868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a22e00-6cb2-4807-a5e8-a049d950233d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:09.414317 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a22e00-6cb2-4807-a5e8-a049d950233d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:09.419332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 381/774] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:09.425502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a22e00-6cb2-4807-a5e8-a049d950233d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:09.459500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:09.486859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2869eb4-5b17-4fe7-b1fc-9e09d1389e60 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:09.542015 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a22e00-6cb2-4807-a5e8-a049d950233d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:09.561171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59c95c-e083-44b3-971b-e98a26d35007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:09.561914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59c95c-e083-44b3-971b-e98a26d35007 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:09.562208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59c95c-e083-44b3-971b-e98a26d35007 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:09.562486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59c95c-e083-44b3-971b-e98a26d35007 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:09.564847 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59c95c-e083-44b3-971b-e98a26d35007 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:09.565172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59c95c-e083-44b3-971b-e98a26d35007 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:09.566596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59c95c-e083-44b3-971b-e98a26d35007 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:09.577832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 394/775] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Jul 25 08:33:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:09.583581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59c95c-e083-44b3-971b-e98a26d35007 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:09.605298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1e6f6-50f1-496a-a2d3-5987f3899e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:09.605989 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1e6f6-50f1-496a-a2d3-5987f3899e51 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:09.606277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1e6f6-50f1-496a-a2d3-5987f3899e51 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:09.606557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1e6f6-50f1-496a-a2d3-5987f3899e51 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:09.607280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1e6f6-50f1-496a-a2d3-5987f3899e51 None admin] RBAC: Adding query filter params (type=network) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:09.607587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1e6f6-50f1-496a-a2d3-5987f3899e51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:09.608999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1e6f6-50f1-496a-a2d3-5987f3899e51 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:09.619024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 382/776] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:09.629598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1e6f6-50f1-496a-a2d3-5987f3899e51 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:09.683002 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:09.700845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59c95c-e083-44b3-971b-e98a26d35007 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:09.736450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4218c5-82e9-4e20-b34b-cb22713546f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:09.737838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4218c5-82e9-4e20-b34b-cb22713546f2 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:09.738174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4218c5-82e9-4e20-b34b-cb22713546f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:09.738457 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4218c5-82e9-4e20-b34b-cb22713546f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:09.739266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4218c5-82e9-4e20-b34b-cb22713546f2 None admin] RBAC: Adding query filter params (interface=public, service_id=fce7fbe2e2c642948f80a9ab9d2019f4, region_id=RegionOne) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:09.739579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4218c5-82e9-4e20-b34b-cb22713546f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:09.740990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4218c5-82e9-4e20-b34b-cb22713546f2 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:09.749522 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 395/777] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Jul 25 08:33:09 2022] GET /identity/v3/endpoints?service_id=fce7fbe2e2c642948f80a9ab9d2019f4&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:09.761739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4218c5-82e9-4e20-b34b-cb22713546f2 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:09.789726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1e6f6-50f1-496a-a2d3-5987f3899e51 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:09.795985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:09.825438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2dbb3-3d14-42cb-9501-88eb0bc75f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:09.826110 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2dbb3-3d14-42cb-9501-88eb0bc75f87 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:09.826399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2dbb3-3d14-42cb-9501-88eb0bc75f87 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:09.826676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2dbb3-3d14-42cb-9501-88eb0bc75f87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:09.827470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2dbb3-3d14-42cb-9501-88eb0bc75f87 None admin] RBAC: Adding query filter params (interface=public, service_id=d4cb84d38ee94b8f8fb4400e57233dd2, region_id=RegionOne) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:09.827782 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2dbb3-3d14-42cb-9501-88eb0bc75f87 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:09.829185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2dbb3-3d14-42cb-9501-88eb0bc75f87 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:09.837350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 383/778] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Jul 25 08:33:09 2022] GET /identity/v3/endpoints?service_id=d4cb84d38ee94b8f8fb4400e57233dd2&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:09.842337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2dbb3-3d14-42cb-9501-88eb0bc75f87 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:09.957975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4218c5-82e9-4e20-b34b-cb22713546f2 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:10.000011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2702d0f-9bfc-4e80-a596-e84783596872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:10.000717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2702d0f-9bfc-4e80-a596-e84783596872 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:10.000998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2702d0f-9bfc-4e80-a596-e84783596872 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:10.001280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2702d0f-9bfc-4e80-a596-e84783596872 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:10.001974 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2702d0f-9bfc-4e80-a596-e84783596872 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:10.003379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2702d0f-9bfc-4e80-a596-e84783596872 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:10.009282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 396/779] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Jul 25 08:33:09 2022] GET /identity/v3/services => generated 772 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:10.030659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2dbb3-3d14-42cb-9501-88eb0bc75f87 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:10.056984 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:10.073111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338afff-91a3-4970-b988-a06c7581c8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:10.073940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338afff-91a3-4970-b988-a06c7581c8ae None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:10.074228 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338afff-91a3-4970-b988-a06c7581c8ae None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:10.074498 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338afff-91a3-4970-b988-a06c7581c8ae None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:10.075264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338afff-91a3-4970-b988-a06c7581c8ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:10.076697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338afff-91a3-4970-b988-a06c7581c8ae None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:10.089319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 384/780] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Jul 25 08:33:09 2022] GET /identity/v3/services => generated 772 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:10.546627 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78dc65-74e1-4893-a972-a61a98182ac8 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:10.547166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78dc65-74e1-4893-a972-a61a98182ac8 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:10.547166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78dc65-74e1-4893-a972-a61a98182ac8 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:10.548896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 397/781] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:10.554246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb47a49-c2d5-4321-9705-345a1b863e95 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:10.554648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb47a49-c2d5-4321-9705-345a1b863e95 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:10.554842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb47a49-c2d5-4321-9705-345a1b863e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:10.589689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-dfb47a49-c2d5-4321-9705-345a1b863e95 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:10.590827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-dfb47a49-c2d5-4321-9705-345a1b863e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:10.679239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 385/782] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:10.709274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805932e-0f86-4577-b1fc-300c7dd55c28 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:10.709604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805932e-0f86-4577-b1fc-300c7dd55c28 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:10.709847 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805932e-0f86-4577-b1fc-300c7dd55c28 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:10.711531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 398/783] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:10.715035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb47a49-c2d5-4321-9705-345a1b863e95 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:10.716069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b228f8a-36a7-4fe5-9247-ca7dd3b8f5d3 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:10.716363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b228f8a-36a7-4fe5-9247-ca7dd3b8f5d3 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:10.716675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b228f8a-36a7-4fe5-9247-ca7dd3b8f5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:10.754136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-8b228f8a-36a7-4fe5-9247-ca7dd3b8f5d3 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:10.755694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8b228f8a-36a7-4fe5-9247-ca7dd3b8f5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:10.839376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:10.861917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb47a49-c2d5-4321-9705-345a1b863e95 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:10.909220 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 399/784] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:10.917921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a52c7-608a-49e9-818e-56f81f963171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:10.918981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a52c7-608a-49e9-818e-56f81f963171 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:10.919462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a52c7-608a-49e9-818e-56f81f963171 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:10.919955 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a52c7-608a-49e9-818e-56f81f963171 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:10.934277 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b228f8a-36a7-4fe5-9247-ca7dd3b8f5d3 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:10.948774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a52c7-608a-49e9-818e-56f81f963171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-a10a52c7-608a-49e9-818e-56f81f963171 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:10.957156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:10.958056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:10.958874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:10.958874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:10.958874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:10.958874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:10.958874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:10.958874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:10.958874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:10.958874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:10.958874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:10.958874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:10.958874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:33:10.958874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:10.959569 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:10.962391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 386/785] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:33:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:10.968324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a52c7-608a-49e9-818e-56f81f963171 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:11.089548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a52c7-608a-49e9-818e-56f81f963171 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:11.099553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b228f8a-36a7-4fe5-9247-ca7dd3b8f5d3 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:11.136263 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8dd202-d28c-4d0b-ae97-9137698e0989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:11.137829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dd202-d28c-4d0b-ae97-9137698e0989 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:11.138633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c913c6-dba6-4628-9555-3370b8438d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:11.139659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c913c6-dba6-4628-9555-3370b8438d02 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:11.140142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dd202-d28c-4d0b-ae97-9137698e0989 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:11.140637 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dd202-d28c-4d0b-ae97-9137698e0989 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:11.141608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8dd202-d28c-4d0b-ae97-9137698e0989 None admin] RBAC: Adding query filter params (name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:11.142095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c913c6-dba6-4628-9555-3370b8438d02 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:11.142557 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c913c6-dba6-4628-9555-3370b8438d02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:11.143598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8dd202-d28c-4d0b-ae97-9137698e0989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:11.145212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8dd202-d28c-4d0b-ae97-9137698e0989 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:11.169150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c913c6-dba6-4628-9555-3370b8438d02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:11.169779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 387/786] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:33:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-c7c913c6-dba6-4628-9555-3370b8438d02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:11.178385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:11.179301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:11.180103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:11.180103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:11.180103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:11.180103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:11.180103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:11.180103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:11.180103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:11.180103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:11.180103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:11.180103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:11.180103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:11.180103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:11.182702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 400/787] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:33:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:11.191187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8dd202-d28c-4d0b-ae97-9137698e0989 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:11.191338 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c913c6-dba6-4628-9555-3370b8438d02 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:11.208916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:11.302721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:11.318284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8dd202-d28c-4d0b-ae97-9137698e0989 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:11.336499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c913c6-dba6-4628-9555-3370b8438d02 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:11.354887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deadcd9-9912-4032-b431-3fef37e00445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:11.355801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deadcd9-9912-4032-b431-3fef37e00445 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:11.356093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deadcd9-9912-4032-b431-3fef37e00445 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:11.356375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deadcd9-9912-4032-b431-3fef37e00445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:11.368451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deadcd9-9912-4032-b431-3fef37e00445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:11.370032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deadcd9-9912-4032-b431-3fef37e00445 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:11.373944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f55aa-1d25-499b-bf3d-38e9eaf5d8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:11.374622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f55aa-1d25-499b-bf3d-38e9eaf5d8c5 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:11.374910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f55aa-1d25-499b-bf3d-38e9eaf5d8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:11.375192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f55aa-1d25-499b-bf3d-38e9eaf5d8c5 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:11.376019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f55aa-1d25-499b-bf3d-38e9eaf5d8c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:11.376331 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f55aa-1d25-499b-bf3d-38e9eaf5d8c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:11.377906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f55aa-1d25-499b-bf3d-38e9eaf5d8c5 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:11.386983 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 401/788] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:33:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-1deadcd9-9912-4032-b431-3fef37e00445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:11.388177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:11.389044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:11.390776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:11.390776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:11.390776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:11.390776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:11.391278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ef832-21e9-4692-a8e0-cdf4a4e3c055 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:11.391278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ef832-21e9-4692-a8e0-cdf4a4e3c055 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:11.391278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ef832-21e9-4692-a8e0-cdf4a4e3c055 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:11.394601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 388/790] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:11.395009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 402/790] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:11.395843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deadcd9-9912-4032-b431-3fef37e00445 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:11.396473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c6952-4224-4fce-9261-a007c28bfe70 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:11.396877 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c6952-4224-4fce-9261-a007c28bfe70 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:11.397232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c6952-4224-4fce-9261-a007c28bfe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:11.417068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-899c6952-4224-4fce-9261-a007c28bfe70 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:11.418429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-899c6952-4224-4fce-9261-a007c28bfe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:11.460323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:11.500640 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 403/791] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:11.501904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c6952-4224-4fce-9261-a007c28bfe70 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:11.506330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deadcd9-9912-4032-b431-3fef37e00445 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:11.547588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2664e4-5d7c-4a60-a7e7-03451dde0f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:11.548400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2664e4-5d7c-4a60-a7e7-03451dde0f31 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:11.548761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2664e4-5d7c-4a60-a7e7-03451dde0f31 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:11.549051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2664e4-5d7c-4a60-a7e7-03451dde0f31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:11.561248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2664e4-5d7c-4a60-a7e7-03451dde0f31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:11.562891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2664e4-5d7c-4a60-a7e7-03451dde0f31 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-5d2664e4-5d7c-4a60-a7e7-03451dde0f31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:11.582704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:11.583655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:11.584533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:11.584533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:11.584533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:11.584533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:11.584533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 389/792] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:11.585270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2664e4-5d7c-4a60-a7e7-03451dde0f31 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:11.590973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:11.630781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c6952-4224-4fce-9261-a007c28bfe70 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:11.667203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264371b4-e406-44b2-85f0-1d7f91ddd6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:11.668734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264371b4-e406-44b2-85f0-1d7f91ddd6ec None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:11.669232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264371b4-e406-44b2-85f0-1d7f91ddd6ec None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:11.669960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264371b4-e406-44b2-85f0-1d7f91ddd6ec None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:11.671051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264371b4-e406-44b2-85f0-1d7f91ddd6ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:11.671584 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264371b4-e406-44b2-85f0-1d7f91ddd6ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:11.673944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264371b4-e406-44b2-85f0-1d7f91ddd6ec None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:11.689392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 404/793] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:11.690594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264371b4-e406-44b2-85f0-1d7f91ddd6ec None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:11.704287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2664e4-5d7c-4a60-a7e7-03451dde0f31 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:11.718556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:11.746229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e90fd7-4ae3-4e0b-8445-7bb96f65abdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:11.747085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e90fd7-4ae3-4e0b-8445-7bb96f65abdc None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:11.747376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e90fd7-4ae3-4e0b-8445-7bb96f65abdc None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:11.747657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e90fd7-4ae3-4e0b-8445-7bb96f65abdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:11.759678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e90fd7-4ae3-4e0b-8445-7bb96f65abdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-f7e90fd7-4ae3-4e0b-8445-7bb96f65abdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:11.768399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:11.769826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:11.770674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:11.770674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:11.770674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:11.770674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:11.770674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:11.770674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:11.770674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:11.770674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:11.770674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:11.770674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:11.770674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:11.770674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:11.775652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 390/794] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:33:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:11.776248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e90fd7-4ae3-4e0b-8445-7bb96f65abdc None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:11.817242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264371b4-e406-44b2-85f0-1d7f91ddd6ec None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:11.843419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:11.857209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec4a8e-05f9-446b-b7ad-00ed9286ec26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:11.858135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec4a8e-05f9-446b-b7ad-00ed9286ec26 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:11.858250 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec4a8e-05f9-446b-b7ad-00ed9286ec26 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:11.858532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec4a8e-05f9-446b-b7ad-00ed9286ec26 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:11.859279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec4a8e-05f9-446b-b7ad-00ed9286ec26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:11.859589 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec4a8e-05f9-446b-b7ad-00ed9286ec26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:11.860999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec4a8e-05f9-446b-b7ad-00ed9286ec26 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:11.878667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 405/795] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:11.879550 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec4a8e-05f9-446b-b7ad-00ed9286ec26 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:11.900112 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e90fd7-4ae3-4e0b-8445-7bb96f65abdc None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:11.943802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e1b7d-f4a4-422d-bca0-0cd727342f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:11.945494 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e1b7d-f4a4-422d-bca0-0cd727342f03 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:11.945937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e1b7d-f4a4-422d-bca0-0cd727342f03 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:11.946639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e1b7d-f4a4-422d-bca0-0cd727342f03 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:11.958575 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e1b7d-f4a4-422d-bca0-0cd727342f03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:11.960262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e1b7d-f4a4-422d-bca0-0cd727342f03 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-7f3e1b7d-f4a4-422d-bca0-0cd727342f03 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:11.979636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:11.980842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:11.981812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:11.981812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:11.981812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:11.981812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:11.981812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:11.981812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:11.981812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:11.981812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:11.981812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 08:33:11.981812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:11.982354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 391/796] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Jul 25 08:33:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:11.983112 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e1b7d-f4a4-422d-bca0-0cd727342f03 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:11.999317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec4a8e-05f9-446b-b7ad-00ed9286ec26 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:12.017709 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:12.045920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662af41a-8970-4e45-a62e-fa558b50c5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:12.047437 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662af41a-8970-4e45-a62e-fa558b50c5b9 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.047973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662af41a-8970-4e45-a62e-fa558b50c5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.048751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662af41a-8970-4e45-a62e-fa558b50c5b9 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.049971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662af41a-8970-4e45-a62e-fa558b50c5b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:12.050799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662af41a-8970-4e45-a62e-fa558b50c5b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:12.052883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662af41a-8970-4e45-a62e-fa558b50c5b9 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:12.064970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 406/797] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:33:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:12.066617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662af41a-8970-4e45-a62e-fa558b50c5b9 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:12.099410 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e1b7d-f4a4-422d-bca0-0cd727342f03 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:12.128966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:12.147211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724e462-612a-49d5-8e99-3e1d1a1f0ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:12.148681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724e462-612a-49d5-8e99-3e1d1a1f0ab1 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.149184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724e462-612a-49d5-8e99-3e1d1a1f0ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.149926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724e462-612a-49d5-8e99-3e1d1a1f0ab1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.161422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724e462-612a-49d5-8e99-3e1d1a1f0ab1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:12.163091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724e462-612a-49d5-8e99-3e1d1a1f0ab1 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-0724e462-612a-49d5-8e99-3e1d1a1f0ab1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:12.183242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:12.184217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:12.185187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:12.185187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:12.185187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:12.185187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:12.185187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:12.185187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:12.185187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:12.185187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:12.185187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:33:12.185187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:12.188669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 392/798] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Jul 25 08:33:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:12.189712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724e462-612a-49d5-8e99-3e1d1a1f0ab1 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:12.200214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662af41a-8970-4e45-a62e-fa558b50c5b9 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:12.236344 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c3eb1-6479-40cd-ba23-af81611c5bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:12.237578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c3eb1-6479-40cd-ba23-af81611c5bde None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.237873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c3eb1-6479-40cd-ba23-af81611c5bde None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.238157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c3eb1-6479-40cd-ba23-af81611c5bde None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.239005 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c3eb1-6479-40cd-ba23-af81611c5bde None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:12.239321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c3eb1-6479-40cd-ba23-af81611c5bde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:12.240780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c3eb1-6479-40cd-ba23-af81611c5bde None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:12.260917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 407/799] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Jul 25 08:33:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:12.261832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c3eb1-6479-40cd-ba23-af81611c5bde None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:12.320288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:12.329889 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724e462-612a-49d5-8e99-3e1d1a1f0ab1 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:12.367023 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf248c-4c50-457c-a6d2-dcb300afd540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:12.368191 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf248c-4c50-457c-a6d2-dcb300afd540 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.368741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf248c-4c50-457c-a6d2-dcb300afd540 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.369227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf248c-4c50-457c-a6d2-dcb300afd540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.380170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c3eb1-6479-40cd-ba23-af81611c5bde None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:12.381594 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf248c-4c50-457c-a6d2-dcb300afd540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:12.383288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf248c-4c50-457c-a6d2-dcb300afd540 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:12.392941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-09cf248c-4c50-457c-a6d2-dcb300afd540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:12.402914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:12.403829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:12.404746 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:12.404746 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:12.404746 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:12.404746 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:12.406969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 393/800] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:12.407996 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf248c-4c50-457c-a6d2-dcb300afd540 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:12.421712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd2522-f22f-421d-ac91-3893cd3cd38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:12.422403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd2522-f22f-421d-ac91-3893cd3cd38d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.422700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd2522-f22f-421d-ac91-3893cd3cd38d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.422988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd2522-f22f-421d-ac91-3893cd3cd38d None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.423839 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd2522-f22f-421d-ac91-3893cd3cd38d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:12.424208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd2522-f22f-421d-ac91-3893cd3cd38d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:12.426266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd2522-f22f-421d-ac91-3893cd3cd38d None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:12.451198 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 408/801] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Jul 25 08:33:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:12.452170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd2522-f22f-421d-ac91-3893cd3cd38d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:12.518637 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf248c-4c50-457c-a6d2-dcb300afd540 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:12.557163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cf9e7-d713-41f0-b1ac-6bb85fb27501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:12.557983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cf9e7-d713-41f0-b1ac-6bb85fb27501 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.558278 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cf9e7-d713-41f0-b1ac-6bb85fb27501 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.558558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cf9e7-d713-41f0-b1ac-6bb85fb27501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.561013 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd2522-f22f-421d-ac91-3893cd3cd38d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:12.569264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cf9e7-d713-41f0-b1ac-6bb85fb27501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:12.570705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cf9e7-d713-41f0-b1ac-6bb85fb27501 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:12.573249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-d98cf9e7-d713-41f0-b1ac-6bb85fb27501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:12.588525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:12.589411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:12.594820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:12.594820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:12.594820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:12.594820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:12.594820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 394/802] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:12.594820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93679af-9b13-4d22-99f0-f272d7f640fb None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.594820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93679af-9b13-4d22-99f0-f272d7f640fb None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.594820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93679af-9b13-4d22-99f0-f272d7f640fb None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.594820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 395/803] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:12.595308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93679af-9b13-4d22-99f0-f272d7f640fb None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:12.598379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed16d8a-0cd8-4ffb-a1ad-56301f82c787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:12.599061 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed16d8a-0cd8-4ffb-a1ad-56301f82c787 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.599350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed16d8a-0cd8-4ffb-a1ad-56301f82c787 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.599631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed16d8a-0cd8-4ffb-a1ad-56301f82c787 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.600364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed16d8a-0cd8-4ffb-a1ad-56301f82c787 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:12.600705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed16d8a-0cd8-4ffb-a1ad-56301f82c787 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:12.602075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed16d8a-0cd8-4ffb-a1ad-56301f82c787 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:12.614142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 409/804] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:12.615388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f92a9-ded9-4c15-9283-5d61c9e56076 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.615678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f92a9-ded9-4c15-9283-5d61c9e56076 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.615961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f92a9-ded9-4c15-9283-5d61c9e56076 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.617487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 410/805] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:12.618253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f92a9-ded9-4c15-9283-5d61c9e56076 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:12.641861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:12.702226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93679af-9b13-4d22-99f0-f272d7f640fb None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:12.718475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f92a9-ded9-4c15-9283-5d61c9e56076 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:12.735532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f00bf9-ae97-4ad5-8c3e-77062789ac7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:12.736176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f00bf9-ae97-4ad5-8c3e-77062789ac7a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.736457 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f00bf9-ae97-4ad5-8c3e-77062789ac7a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.736761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f00bf9-ae97-4ad5-8c3e-77062789ac7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.754111 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38562ec-921e-493e-85b9-53b20e25ff58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:12.755023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38562ec-921e-493e-85b9-53b20e25ff58 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.755313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38562ec-921e-493e-85b9-53b20e25ff58 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.755591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38562ec-921e-493e-85b9-53b20e25ff58 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.756299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38562ec-921e-493e-85b9-53b20e25ff58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:12.756635 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38562ec-921e-493e-85b9-53b20e25ff58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:12.757981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38562ec-921e-493e-85b9-53b20e25ff58 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:12.773454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 411/806] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:12.773454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09594a8f-b0cb-4425-99e8-f48d25402420 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.773454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09594a8f-b0cb-4425-99e8-f48d25402420 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.773454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09594a8f-b0cb-4425-99e8-f48d25402420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.788585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-09594a8f-b0cb-4425-99e8-f48d25402420 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:12.789798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-09594a8f-b0cb-4425-99e8-f48d25402420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:12.795942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:12.803744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f00bf9-ae97-4ad5-8c3e-77062789ac7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:12.805217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f00bf9-ae97-4ad5-8c3e-77062789ac7a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:12.855593 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 412/807] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:12.856473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09594a8f-b0cb-4425-99e8-f48d25402420 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:12.871072 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:12.905188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 396/808] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:33:12.906790 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d408ba-3e36-4139-a2b8-4374c759f7a1 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.907087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d408ba-3e36-4139-a2b8-4374c759f7a1 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.907368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d408ba-3e36-4139-a2b8-4374c759f7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:12.925235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-10d408ba-3e36-4139-a2b8-4374c759f7a1 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:12.926542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-10d408ba-3e36-4139-a2b8-4374c759f7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:12.961841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09594a8f-b0cb-4425-99e8-f48d25402420 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:12.991461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 397/809] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:12.992352 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d408ba-3e36-4139-a2b8-4374c759f7a1 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:12.995870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db762779-8fd5-4100-996d-df1393a3dfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:12.996719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db762779-8fd5-4100-996d-df1393a3dfdf None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:12.997012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db762779-8fd5-4100-996d-df1393a3dfdf None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:12.997293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db762779-8fd5-4100-996d-df1393a3dfdf None admin] PATH_INFO: `/v3/users/swift` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.013902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db762779-8fd5-4100-996d-df1393a3dfdf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.015440 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db762779-8fd5-4100-996d-df1393a3dfdf None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.017458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-db762779-8fd5-4100-996d-df1393a3dfdf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:13.031278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:13.032264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:13.033234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:13.033234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:13.033234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:13.033234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:13.033234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:13.033234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:13.033234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:13.033234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:13.033234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 08:33:13.033234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:13.033234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 413/810] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:33:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:13.033827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db762779-8fd5-4100-996d-df1393a3dfdf None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:13.094791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d408ba-3e36-4139-a2b8-4374c759f7a1 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:13.103097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:13.128343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1830-6ba5-4415-8c8b-3afadd99ce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:13.129174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1830-6ba5-4415-8c8b-3afadd99ce1b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:13.129460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1830-6ba5-4415-8c8b-3afadd99ce1b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:13.129737 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1830-6ba5-4415-8c8b-3afadd99ce1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.132415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db762779-8fd5-4100-996d-df1393a3dfdf None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:13.136481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e1830-6ba5-4415-8c8b-3afadd99ce1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.137936 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e1830-6ba5-4415-8c8b-3afadd99ce1b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-f05e1830-6ba5-4415-8c8b-3afadd99ce1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:13.152089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:13.153359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:13.154272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:13.154272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:13.154272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:13.154272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:13.154272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:13.154272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:13.154272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:13.154272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:13.156718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 398/811] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:13.157715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1830-6ba5-4415-8c8b-3afadd99ce1b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:13.166466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05278764-673f-46b7-aacb-52220f9c984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:13.167264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05278764-673f-46b7-aacb-52220f9c984f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:13.167555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05278764-673f-46b7-aacb-52220f9c984f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:13.167835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05278764-673f-46b7-aacb-52220f9c984f None admin] PATH_INFO: `/v3/services/network` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.168491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05278764-673f-46b7-aacb-52220f9c984f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.169901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05278764-673f-46b7-aacb-52220f9c984f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-05278764-673f-46b7-aacb-52220f9c984f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:13.181611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:13.182748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 08:33:13.183745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:13.187469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 414/812] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:13 2022] GET /identity/v3/services/network => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:13.188231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05278764-673f-46b7-aacb-52220f9c984f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:13.241006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:13.256549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1830-6ba5-4415-8c8b-3afadd99ce1b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:13.290856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05278764-673f-46b7-aacb-52220f9c984f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:13.292962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c83656-c92e-4b48-9836-80bacb17c14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:13.294587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c83656-c92e-4b48-9836-80bacb17c14e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:13.294874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c83656-c92e-4b48-9836-80bacb17c14e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:13.295151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c83656-c92e-4b48-9836-80bacb17c14e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.308735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c83656-c92e-4b48-9836-80bacb17c14e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.310188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c83656-c92e-4b48-9836-80bacb17c14e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.324300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04ce5f-a697-4d2e-9a8c-b4c3d131418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:13.325143 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04ce5f-a697-4d2e-9a8c-b4c3d131418e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:13.325429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04ce5f-a697-4d2e-9a8c-b4c3d131418e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:13.325703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04ce5f-a697-4d2e-9a8c-b4c3d131418e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.326357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04ce5f-a697-4d2e-9a8c-b4c3d131418e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-52c83656-c92e-4b48-9836-80bacb17c14e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:13.327210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:13.328154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:13.329127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:13.329127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:13.329127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:13.329127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:13.329345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04ce5f-a697-4d2e-9a8c-b4c3d131418e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.329413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 399/813] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:13.329603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c83656-c92e-4b48-9836-80bacb17c14e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:13.333767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-4c04ce5f-a697-4d2e-9a8c-b4c3d131418e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:13.344432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:13.345574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:13.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 415/814] 10.222.0.52 () {56 vars in 1129 bytes} [Mon Jul 25 08:33:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:13.348138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04ce5f-a697-4d2e-9a8c-b4c3d131418e None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:13.434509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c83656-c92e-4b48-9836-80bacb17c14e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:13.446408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04ce5f-a697-4d2e-9a8c-b4c3d131418e None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:13.468347 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96aa25-331d-423a-aec9-d4c5fbbb9282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:13.469041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96aa25-331d-423a-aec9-d4c5fbbb9282 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:13.469322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96aa25-331d-423a-aec9-d4c5fbbb9282 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:13.469599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96aa25-331d-423a-aec9-d4c5fbbb9282 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.470430 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96aa25-331d-423a-aec9-d4c5fbbb9282 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:13.470735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96aa25-331d-423a-aec9-d4c5fbbb9282 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.472684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96aa25-331d-423a-aec9-d4c5fbbb9282 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.479636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f04a9f-209d-4c66-ae5b-c9cef8935f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:13.480365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04a9f-209d-4c66-ae5b-c9cef8935f9e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:13.480676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04a9f-209d-4c66-ae5b-c9cef8935f9e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:13.480954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04a9f-209d-4c66-ae5b-c9cef8935f9e None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.481741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f04a9f-209d-4c66-ae5b-c9cef8935f9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:13.482049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f04a9f-209d-4c66-ae5b-c9cef8935f9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.483492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f04a9f-209d-4c66-ae5b-c9cef8935f9e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.484677 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:13.496140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 400/815] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Jul 25 08:33:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:13.496680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 416/816] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:13.497395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96aa25-331d-423a-aec9-d4c5fbbb9282 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:13.497538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f04a9f-209d-4c66-ae5b-c9cef8935f9e None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:13.560024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:13.599936 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f04a9f-209d-4c66-ae5b-c9cef8935f9e None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:13.600536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96aa25-331d-423a-aec9-d4c5fbbb9282 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:13.633623 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ad2bc-1b49-4e41-a11d-cb5c8adc5a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:13.633800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ce31a-b866-451a-b98d-af08be50d4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:13.634280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ad2bc-1b49-4e41-a11d-cb5c8adc5a96 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:13.634383 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ce31a-b866-451a-b98d-af08be50d4b2 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:13.634563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ce31a-b866-451a-b98d-af08be50d4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:13.634654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ad2bc-1b49-4e41-a11d-cb5c8adc5a96 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:13.634835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ad2bc-1b49-4e41-a11d-cb5c8adc5a96 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.634923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ce31a-b866-451a-b98d-af08be50d4b2 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.635539 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ad2bc-1b49-4e41-a11d-cb5c8adc5a96 None admin] RBAC: Adding query filter params (name=network) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:13.635637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ce31a-b866-451a-b98d-af08be50d4b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:13.635840 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ce31a-b866-451a-b98d-af08be50d4b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.635927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ad2bc-1b49-4e41-a11d-cb5c8adc5a96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.637221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ad2bc-1b49-4e41-a11d-cb5c8adc5a96 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.637313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ce31a-b866-451a-b98d-af08be50d4b2 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.642182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 401/817] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:13.642940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ad2bc-1b49-4e41-a11d-cb5c8adc5a96 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:13.649226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 417/818] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:13.650201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ce31a-b866-451a-b98d-af08be50d4b2 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:13.702802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:13.743583 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ad2bc-1b49-4e41-a11d-cb5c8adc5a96 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:13.749314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ce31a-b866-451a-b98d-af08be50d4b2 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:13.768762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:13.777860 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f0c07-87a8-48a6-8f05-d909bd313690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:13.778528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f0c07-87a8-48a6-8f05-d909bd313690 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:13.778812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f0c07-87a8-48a6-8f05-d909bd313690 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:13.779091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f0c07-87a8-48a6-8f05-d909bd313690 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.779801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f0c07-87a8-48a6-8f05-d909bd313690 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:13.780105 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f0c07-87a8-48a6-8f05-d909bd313690 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.781500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f0c07-87a8-48a6-8f05-d909bd313690 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.783597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf2b57-9199-4e47-8279-763edbd50d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:13.785013 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf2b57-9199-4e47-8279-763edbd50d64 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:13.785296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf2b57-9199-4e47-8279-763edbd50d64 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:13.785573 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf2b57-9199-4e47-8279-763edbd50d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.786457 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 402/819] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Jul 25 08:33:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:13.787200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f0c07-87a8-48a6-8f05-d909bd313690 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:13.796555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf2b57-9199-4e47-8279-763edbd50d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.798025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf2b57-9199-4e47-8279-763edbd50d64 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-04cf2b57-9199-4e47-8279-763edbd50d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:13.814855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:13.815693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:13.821269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:13.821269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:13.821269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:13.821269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:13.821269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 418/820] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:13.821269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf2b57-9199-4e47-8279-763edbd50d64 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:13.888352 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f0c07-87a8-48a6-8f05-d909bd313690 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:13.912796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:13.915300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf2b57-9199-4e47-8279-763edbd50d64 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:13.923556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f808243-cea9-4fab-8ace-6f14ce18a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:13.924488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f808243-cea9-4fab-8ace-6f14ce18a78d None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:13.925009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f808243-cea9-4fab-8ace-6f14ce18a78d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:13.925683 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f808243-cea9-4fab-8ace-6f14ce18a78d None admin] PATH_INFO: `/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/c151c53202f24faeb3accf802716652f/roles/45565d93c8c142e086e6b7ce62ff84e4` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.939473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f808243-cea9-4fab-8ace-6f14ce18a78d None admin] RBAC: Authorizing `identity:create_grant(project_id=2ee27c4da77c47428e02fd65e890d256, user_id=c151c53202f24faeb3accf802716652f, role_id=45565d93c8c142e086e6b7ce62ff84e4)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.940989 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f808243-cea9-4fab-8ace-6f14ce18a78d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.948834 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1fa8b-1fff-4469-aa43-f874d55414cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:13.949497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1fa8b-1fff-4469-aa43-f874d55414cd None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:13.949783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1fa8b-1fff-4469-aa43-f874d55414cd None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:13.950079 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1fa8b-1fff-4469-aa43-f874d55414cd None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:13.950777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1fa8b-1fff-4469-aa43-f874d55414cd None admin] RBAC: Adding query filter params (type=network) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:13.951085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1fa8b-1fff-4469-aa43-f874d55414cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:13.952432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1fa8b-1fff-4469-aa43-f874d55414cd None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:13.953116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 403/821] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:33:13 2022] PUT /identity/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/c151c53202f24faeb3accf802716652f/roles/45565d93c8c142e086e6b7ce62ff84e4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:33:13.953803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f808243-cea9-4fab-8ace-6f14ce18a78d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:13.965899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 419/822] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:13.967358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1fa8b-1fff-4469-aa43-f874d55414cd None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:14.015227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:14.068685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f808243-cea9-4fab-8ace-6f14ce18a78d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:14.093634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1fa8b-1fff-4469-aa43-f874d55414cd None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:14.113464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f0d19-00c6-4236-a268-b20468cff959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:14.114273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f0d19-00c6-4236-a268-b20468cff959 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:14.114620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f0d19-00c6-4236-a268-b20468cff959 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:14.114900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f0d19-00c6-4236-a268-b20468cff959 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:14.121943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f0d19-00c6-4236-a268-b20468cff959 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:14.123457 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f0d19-00c6-4236-a268-b20468cff959 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:14.133985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8475255-b1dc-4a55-b86c-c574c22942e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:14.134749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8475255-b1dc-4a55-b86c-c574c22942e7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:14.135125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8475255-b1dc-4a55-b86c-c574c22942e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:14.135475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8475255-b1dc-4a55-b86c-c574c22942e7 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:14.136260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8475255-b1dc-4a55-b86c-c574c22942e7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:14.136665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8475255-b1dc-4a55-b86c-c574c22942e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-e20f0d19-00c6-4236-a268-b20468cff959 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:14.137966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:14.138835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:14.139721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:14.139721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:14.139721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:14.139721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:14.139721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:14.139721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:14.139721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:14.139721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:14.140148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8475255-b1dc-4a55-b86c-c574c22942e7 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:14.140218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 404/823] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:14.140292 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f0d19-00c6-4236-a268-b20468cff959 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:14.153279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 420/824] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Jul 25 08:33:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:14.153279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8475255-b1dc-4a55-b86c-c574c22942e7 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:14.180267 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:14.249962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f0d19-00c6-4236-a268-b20468cff959 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:14.254649 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:14.257836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8475255-b1dc-4a55-b86c-c574c22942e7 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:14.287501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5978b-3806-44b1-961f-ebd01c948d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:14.288123 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5978b-3806-44b1-961f-ebd01c948d72 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:14.288413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5978b-3806-44b1-961f-ebd01c948d72 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:14.288741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5978b-3806-44b1-961f-ebd01c948d72 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:14.289479 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5978b-3806-44b1-961f-ebd01c948d72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:14.289788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5978b-3806-44b1-961f-ebd01c948d72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:14.291167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5978b-3806-44b1-961f-ebd01c948d72 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:14.298154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d04bf3-34c0-4c2a-9d9a-d3f4fcf6359e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:14.298822 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d04bf3-34c0-4c2a-9d9a-d3f4fcf6359e None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:14.299105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d04bf3-34c0-4c2a-9d9a-d3f4fcf6359e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:14.299381 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d04bf3-34c0-4c2a-9d9a-d3f4fcf6359e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:14.300219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d04bf3-34c0-4c2a-9d9a-d3f4fcf6359e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:14.303359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d04bf3-34c0-4c2a-9d9a-d3f4fcf6359e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:14.308459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 405/825] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:14.309694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5978b-3806-44b1-961f-ebd01c948d72 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:14.324645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 421/826] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:33:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:14.325530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d04bf3-34c0-4c2a-9d9a-d3f4fcf6359e None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:14.415916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:14.423443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5978b-3806-44b1-961f-ebd01c948d72 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:14.471750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d04bf3-34c0-4c2a-9d9a-d3f4fcf6359e None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:14.472138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5aaf4-a926-4c8f-a8a9-349187c37fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:14.472930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5aaf4-a926-4c8f-a8a9-349187c37fcb None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:14.473285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5aaf4-a926-4c8f-a8a9-349187c37fcb None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:14.473634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5aaf4-a926-4c8f-a8a9-349187c37fcb None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:14.474541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5aaf4-a926-4c8f-a8a9-349187c37fcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:14.474943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5aaf4-a926-4c8f-a8a9-349187c37fcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:14.476603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5aaf4-a926-4c8f-a8a9-349187c37fcb None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:14.496106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 406/827] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:14.497467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5aaf4-a926-4c8f-a8a9-349187c37fcb None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:14.515935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b05989-19d9-41cd-857a-4208c6162cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:14.517107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b05989-19d9-41cd-857a-4208c6162cf8 None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:14.517679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b05989-19d9-41cd-857a-4208c6162cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:14.517964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b05989-19d9-41cd-857a-4208c6162cf8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:14.519012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b05989-19d9-41cd-857a-4208c6162cf8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:14.520738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b05989-19d9-41cd-857a-4208c6162cf8 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:14.546443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 422/828] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:33:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:14.547045 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b05989-19d9-41cd-857a-4208c6162cf8 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:14.563623 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:14.626713 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5aaf4-a926-4c8f-a8a9-349187c37fcb None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:14.726636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b05989-19d9-41cd-857a-4208c6162cf8 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:14.756883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e06b42-24b8-4a20-84f5-ff9ee67e182b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:14.757707 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e06b42-24b8-4a20-84f5-ff9ee67e182b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:14.757997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e06b42-24b8-4a20-84f5-ff9ee67e182b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:14.758274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e06b42-24b8-4a20-84f5-ff9ee67e182b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:14.767409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e06b42-24b8-4a20-84f5-ff9ee67e182b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:14.767733 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c3c68-f26c-4998-a1c1-750d801d90bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:14.768443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c3c68-f26c-4998-a1c1-750d801d90bf None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:14.768807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c3c68-f26c-4998-a1c1-750d801d90bf None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:14.769031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e06b42-24b8-4a20-84f5-ff9ee67e182b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:14.769147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c3c68-f26c-4998-a1c1-750d801d90bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:14.769934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c3c68-f26c-4998-a1c1-750d801d90bf None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=8b476d6b82ae464b81fc8498dc0e68c7) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:14.770307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c3c68-f26c-4998-a1c1-750d801d90bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:14.771752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c3c68-f26c-4998-a1c1-750d801d90bf None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:14.779941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 423/829] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:14 2022] GET /identity/v3/role_assignments?user.id=8b476d6b82ae464b81fc8498dc0e68c7&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-b6e06b42-24b8-4a20-84f5-ff9ee67e182b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:14.797672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:14.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:14.800154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:14.800154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:14.800154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:14.800154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:14.800154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:14.800154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:14.800154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:14.800154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:14.800684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 407/830] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:14.806711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c3c68-f26c-4998-a1c1-750d801d90bf None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:14.890877 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:14.932684 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c3c68-f26c-4998-a1c1-750d801d90bf None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:14.981176 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351bc44-3f19-4f97-b362-ffe067c8ff6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:14.982205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351bc44-3f19-4f97-b362-ffe067c8ff6f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:14.982715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351bc44-3f19-4f97-b362-ffe067c8ff6f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:14.983204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351bc44-3f19-4f97-b362-ffe067c8ff6f None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:14.984189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351bc44-3f19-4f97-b362-ffe067c8ff6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:14.984719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351bc44-3f19-4f97-b362-ffe067c8ff6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:14.986403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351bc44-3f19-4f97-b362-ffe067c8ff6f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:15.002894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 424/831] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:15.009212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e06b42-24b8-4a20-84f5-ff9ee67e182b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:15.032470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:15.147703 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e06b42-24b8-4a20-84f5-ff9ee67e182b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:15.183118 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f6b3a-4a26-4d53-81d6-343b743802de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:15.184009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f6b3a-4a26-4d53-81d6-343b743802de None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:15.184293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f6b3a-4a26-4d53-81d6-343b743802de None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:15.184589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f6b3a-4a26-4d53-81d6-343b743802de None admin] PATH_INFO: `/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/7d6be43abd85417c9fb7bbcb36a1f77d/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:15.186592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f6b3a-4a26-4d53-81d6-343b743802de None admin] RBAC: Authorizing `identity:create_grant(project_id=2ee27c4da77c47428e02fd65e890d256, user_id=7d6be43abd85417c9fb7bbcb36a1f77d, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:15.188147 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f6b3a-4a26-4d53-81d6-343b743802de None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:15.208761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 408/832] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:33:15 2022] PUT /identity/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/7d6be43abd85417c9fb7bbcb36a1f77d/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:33:15.980101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c0413-d29c-4458-b159-83b31d12b1ac None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:15.980642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c0413-d29c-4458-b159-83b31d12b1ac None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:15.980724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c0413-d29c-4458-b159-83b31d12b1ac None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:15.982284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 425/833] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:15.988126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d4db1-a15e-4582-8807-376f54224b15 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:15.988532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d4db1-a15e-4582-8807-376f54224b15 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:15.988722 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d4db1-a15e-4582-8807-376f54224b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:16.008413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-4a0d4db1-a15e-4582-8807-376f54224b15 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:16.009625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4a0d4db1-a15e-4582-8807-376f54224b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:16.074923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:16.080242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 409/834] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:16.105073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c0413-d29c-4458-b159-83b31d12b1ac None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:16.207803 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c0413-d29c-4458-b159-83b31d12b1ac None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:16.234199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:16.243618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde5987-14c7-4c2a-9374-20bff4429c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:16.244548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde5987-14c7-4c2a-9374-20bff4429c0b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:16.244871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde5987-14c7-4c2a-9374-20bff4429c0b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:16.245147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde5987-14c7-4c2a-9374-20bff4429c0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:16.256113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde5987-14c7-4c2a-9374-20bff4429c0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-7dde5987-14c7-4c2a-9374-20bff4429c0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:16.263837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:16.264776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:16.265609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 426/835] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:33:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:16.271188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d4db1-a15e-4582-8807-376f54224b15 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:16.377244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d4db1-a15e-4582-8807-376f54224b15 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:16.412375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3ef44-8cee-4fa2-9e0e-9ccb77411781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:16.413080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3ef44-8cee-4fa2-9e0e-9ccb77411781 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:16.413364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3ef44-8cee-4fa2-9e0e-9ccb77411781 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:16.413635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3ef44-8cee-4fa2-9e0e-9ccb77411781 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:16.414382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3ef44-8cee-4fa2-9e0e-9ccb77411781 None admin] RBAC: Adding query filter params (name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:16.414687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3ef44-8cee-4fa2-9e0e-9ccb77411781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:16.416046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3ef44-8cee-4fa2-9e0e-9ccb77411781 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:16.425180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 410/836] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:33:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:16.430742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde5987-14c7-4c2a-9374-20bff4429c0b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:16.535945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde5987-14c7-4c2a-9374-20bff4429c0b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:16.571439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf9f00-0a6f-40ce-ad56-6ba1f9e48131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:16.572582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf9f00-0a6f-40ce-ad56-6ba1f9e48131 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:16.573070 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf9f00-0a6f-40ce-ad56-6ba1f9e48131 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:16.573545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf9f00-0a6f-40ce-ad56-6ba1f9e48131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:16.584903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf9f00-0a6f-40ce-ad56-6ba1f9e48131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:16.586641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf9f00-0a6f-40ce-ad56-6ba1f9e48131 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-d4bf9f00-0a6f-40ce-ad56-6ba1f9e48131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:16.605091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:16.605996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:16.606876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:16.606876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:16.606876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:16.606876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:16.609092 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 427/837] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:16.615088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3ef44-8cee-4fa2-9e0e-9ccb77411781 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:16.684407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:16.723506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3ef44-8cee-4fa2-9e0e-9ccb77411781 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:16.757822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a9ab3-d30c-4674-9ec5-8e407daf2628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:16.758498 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a9ab3-d30c-4674-9ec5-8e407daf2628 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:16.758782 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a9ab3-d30c-4674-9ec5-8e407daf2628 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:16.759062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a9ab3-d30c-4674-9ec5-8e407daf2628 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:16.759789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a9ab3-d30c-4674-9ec5-8e407daf2628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:16.760099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a9ab3-d30c-4674-9ec5-8e407daf2628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:16.761497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a9ab3-d30c-4674-9ec5-8e407daf2628 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:16.774502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 411/838] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:16.779927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf9f00-0a6f-40ce-ad56-6ba1f9e48131 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:16.823925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:16.831783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238ac30-72a3-46be-b63b-b59ff0f13062 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:16.832080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238ac30-72a3-46be-b63b-b59ff0f13062 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:16.832367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238ac30-72a3-46be-b63b-b59ff0f13062 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:16.833910 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 412/839] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:16.838881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd026a9c-4f91-4029-a6b2-8e9a1661b4f3 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:16.839175 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd026a9c-4f91-4029-a6b2-8e9a1661b4f3 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:16.839457 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd026a9c-4f91-4029-a6b2-8e9a1661b4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:16.858166 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-fd026a9c-4f91-4029-a6b2-8e9a1661b4f3 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:16.859573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-fd026a9c-4f91-4029-a6b2-8e9a1661b4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:16.891435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf9f00-0a6f-40ce-ad56-6ba1f9e48131 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:16.928791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230fec-2747-40d3-90d5-aa6d80937221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:16.929947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230fec-2747-40d3-90d5-aa6d80937221 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:16.930239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230fec-2747-40d3-90d5-aa6d80937221 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:16.930656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230fec-2747-40d3-90d5-aa6d80937221 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:16.938507 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 413/840] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:16.941680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb230fec-2747-40d3-90d5-aa6d80937221 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:16.943335 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb230fec-2747-40d3-90d5-aa6d80937221 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-fb230fec-2747-40d3-90d5-aa6d80937221 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:16.959313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:16.960481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:16.961566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:16.961566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:16.961566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:16.961566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:16.961566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:16.961566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:16.961566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:16.961566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:16.961566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 08:33:16.961566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:16.962590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 428/841] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Jul 25 08:33:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:16.967976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd026a9c-4f91-4029-a6b2-8e9a1661b4f3 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:16.973597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230fec-2747-40d3-90d5-aa6d80937221 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:17.020517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:17.070661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd026a9c-4f91-4029-a6b2-8e9a1661b4f3 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:17.084378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:17.088132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230fec-2747-40d3-90d5-aa6d80937221 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:17.107269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdaaf34-21cd-4e1e-a7a1-ff079c277952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:17.108080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaaf34-21cd-4e1e-a7a1-ff079c277952 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.108368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaaf34-21cd-4e1e-a7a1-ff079c277952 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.108674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaaf34-21cd-4e1e-a7a1-ff079c277952 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.109554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdaaf34-21cd-4e1e-a7a1-ff079c277952 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:17.110845 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdaaf34-21cd-4e1e-a7a1-ff079c277952 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-afdaaf34-21cd-4e1e-a7a1-ff079c277952 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:17.122826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:17.123810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:17.125168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 414/842] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:33:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:17.125974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b9ce3-0ea7-40eb-a584-b6273d9f48b4 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.125974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b9ce3-0ea7-40eb-a584-b6273d9f48b4 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.125974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b9ce3-0ea7-40eb-a584-b6273d9f48b4 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.127167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 415/843] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:17.130041 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877b166-c02e-4d42-94a6-d1a8ec25f05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:17.130956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877b166-c02e-4d42-94a6-d1a8ec25f05d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.131433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877b166-c02e-4d42-94a6-d1a8ec25f05d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.131917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877b166-c02e-4d42-94a6-d1a8ec25f05d None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.132157 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b9ce3-0ea7-40eb-a584-b6273d9f48b4 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:17.133308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877b166-c02e-4d42-94a6-d1a8ec25f05d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:17.133828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877b166-c02e-4d42-94a6-d1a8ec25f05d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:17.135452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877b166-c02e-4d42-94a6-d1a8ec25f05d None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:17.154744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 429/844] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Jul 25 08:33:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:17.156355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01527f-8ed0-4457-bb34-c309f5ce94cc None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.156745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01527f-8ed0-4457-bb34-c309f5ce94cc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.157092 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01527f-8ed0-4457-bb34-c309f5ce94cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.176178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-ab01527f-8ed0-4457-bb34-c309f5ce94cc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:17.177400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ab01527f-8ed0-4457-bb34-c309f5ce94cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:17.244808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b9ce3-0ea7-40eb-a584-b6273d9f48b4 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:17.249023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 430/845] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:17.250083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01527f-8ed0-4457-bb34-c309f5ce94cc None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:17.263325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:17.283013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd85f7-21b5-4027-9a0e-1129f105e136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:17.283726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd85f7-21b5-4027-9a0e-1129f105e136 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.284013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd85f7-21b5-4027-9a0e-1129f105e136 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.284294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd85f7-21b5-4027-9a0e-1129f105e136 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.285039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd85f7-21b5-4027-9a0e-1129f105e136 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:17.285565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd85f7-21b5-4027-9a0e-1129f105e136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:17.286792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd85f7-21b5-4027-9a0e-1129f105e136 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:17.294575 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 416/846] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:33:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:17.295388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd85f7-21b5-4027-9a0e-1129f105e136 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:17.355826 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:17.367521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01527f-8ed0-4457-bb34-c309f5ce94cc None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:17.392808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd85f7-21b5-4027-9a0e-1129f105e136 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:17.401591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b05331-a495-4fdc-b502-529cb5339e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:17.402503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05331-a495-4fdc-b502-529cb5339e47 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.402791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05331-a495-4fdc-b502-529cb5339e47 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.403073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05331-a495-4fdc-b502-529cb5339e47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.427240 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0df54-02ea-41ce-9a2b-fab798d6cf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:17.428048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0df54-02ea-41ce-9a2b-fab798d6cf1a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.428337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0df54-02ea-41ce-9a2b-fab798d6cf1a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.428648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0df54-02ea-41ce-9a2b-fab798d6cf1a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.429313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0df54-02ea-41ce-9a2b-fab798d6cf1a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:17.430676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0df54-02ea-41ce-9a2b-fab798d6cf1a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-10c0df54-02ea-41ce-9a2b-fab798d6cf1a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:17.443548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:17.444625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:17.445451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 417/847] 10.222.0.52 () {56 vars in 1132 bytes} [Mon Jul 25 08:33:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:17.446293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f4e28-5d21-4879-8c95-d4d9130ba2ed None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.446293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f4e28-5d21-4879-8c95-d4d9130ba2ed None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.446520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f4e28-5d21-4879-8c95-d4d9130ba2ed None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.447923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 418/848] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:17.448486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f4e28-5d21-4879-8c95-d4d9130ba2ed None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:17.475301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b05331-a495-4fdc-b502-529cb5339e47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:17.476786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b05331-a495-4fdc-b502-529cb5339e47 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:17.495117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:17.548872 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f4e28-5d21-4879-8c95-d4d9130ba2ed None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:17.561440 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:17.575452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 431/849] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:33:17.576097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b05331-a495-4fdc-b502-529cb5339e47 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:17.587754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b1e58-e1e5-44fa-bd28-5c624606b561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:17.588422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1e58-e1e5-44fa-bd28-5c624606b561 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.588744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1e58-e1e5-44fa-bd28-5c624606b561 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.589024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1e58-e1e5-44fa-bd28-5c624606b561 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.589739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b1e58-e1e5-44fa-bd28-5c624606b561 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:17.590049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b1e58-e1e5-44fa-bd28-5c624606b561 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:17.591416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b1e58-e1e5-44fa-bd28-5c624606b561 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:17.595761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 419/850] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:33:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:17.597010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faad3c9-39ef-4cfb-a171-40f32c902a3b None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.597306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faad3c9-39ef-4cfb-a171-40f32c902a3b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.597592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faad3c9-39ef-4cfb-a171-40f32c902a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.617274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-7faad3c9-39ef-4cfb-a171-40f32c902a3b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:17.618589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7faad3c9-39ef-4cfb-a171-40f32c902a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:17.695476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b05331-a495-4fdc-b502-529cb5339e47 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:17.703938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 420/851] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:17.705221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faad3c9-39ef-4cfb-a171-40f32c902a3b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:17.734156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac7d66-fc5e-493d-9239-3d0c2af8a877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:17.735195 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac7d66-fc5e-493d-9239-3d0c2af8a877 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.735704 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac7d66-fc5e-493d-9239-3d0c2af8a877 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.736197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac7d66-fc5e-493d-9239-3d0c2af8a877 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.737843 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac7d66-fc5e-493d-9239-3d0c2af8a877 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:17.738156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac7d66-fc5e-493d-9239-3d0c2af8a877 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:17.739530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac7d66-fc5e-493d-9239-3d0c2af8a877 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:17.745570 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 432/852] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Jul 25 08:33:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:17.746274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac7d66-fc5e-493d-9239-3d0c2af8a877 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:17.762230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:17.804404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:17.832840 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faad3c9-39ef-4cfb-a171-40f32c902a3b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:17.855683 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac7d66-fc5e-493d-9239-3d0c2af8a877 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:17.871404 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315b50b-0f8b-41f8-a8a2-ebf8c6fb5783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:17.872171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315b50b-0f8b-41f8-a8a2-ebf8c6fb5783 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.872423 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315b50b-0f8b-41f8-a8a2-ebf8c6fb5783 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.872730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315b50b-0f8b-41f8-a8a2-ebf8c6fb5783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.883034 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315b50b-0f8b-41f8-a8a2-ebf8c6fb5783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:17.884470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315b50b-0f8b-41f8-a8a2-ebf8c6fb5783 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:17.892679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51a7e8-fcfc-456d-ac99-cd2dd30d9c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:17.893515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51a7e8-fcfc-456d-ac99-cd2dd30d9c61 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:17.893799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51a7e8-fcfc-456d-ac99-cd2dd30d9c61 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:17.894086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51a7e8-fcfc-456d-ac99-cd2dd30d9c61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-8315b50b-0f8b-41f8-a8a2-ebf8c6fb5783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:17.902568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:17.903512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:17.904409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:17.904409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:17.904409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:17.904409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:17.904409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 421/853] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:17.905199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315b50b-0f8b-41f8-a8a2-ebf8c6fb5783 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:17.910199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51a7e8-fcfc-456d-ac99-cd2dd30d9c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-6e51a7e8-fcfc-456d-ac99-cd2dd30d9c61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:17.917931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:17.918857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:17.919692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 433/854] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:33:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:17.920375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51a7e8-fcfc-456d-ac99-cd2dd30d9c61 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:18.011888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:18.023198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315b50b-0f8b-41f8-a8a2-ebf8c6fb5783 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:18.028504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51a7e8-fcfc-456d-ac99-cd2dd30d9c61 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:18.045753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:18.060352 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdcdfa-a67f-42d9-b07d-3e1baaf6bed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:18.061074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdcdfa-a67f-42d9-b07d-3e1baaf6bed8 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:18.061365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdcdfa-a67f-42d9-b07d-3e1baaf6bed8 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:18.061648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdcdfa-a67f-42d9-b07d-3e1baaf6bed8 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:18.062373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdcdfa-a67f-42d9-b07d-3e1baaf6bed8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:18.062684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdcdfa-a67f-42d9-b07d-3e1baaf6bed8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:18.064059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdcdfa-a67f-42d9-b07d-3e1baaf6bed8 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:18.069537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 422/855] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Jul 25 08:33:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:18.070481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdcdfa-a67f-42d9-b07d-3e1baaf6bed8 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:18.082707 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853c318-0e4b-45f8-bbb9-a8291fa8f085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:18.083583 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853c318-0e4b-45f8-bbb9-a8291fa8f085 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:18.083970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853c318-0e4b-45f8-bbb9-a8291fa8f085 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:18.084352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853c318-0e4b-45f8-bbb9-a8291fa8f085 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:18.085373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853c318-0e4b-45f8-bbb9-a8291fa8f085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:18.085960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853c318-0e4b-45f8-bbb9-a8291fa8f085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:18.087338 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853c318-0e4b-45f8-bbb9-a8291fa8f085 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:18.106307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 434/856] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:18.109680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853c318-0e4b-45f8-bbb9-a8291fa8f085 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:18.219979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdcdfa-a67f-42d9-b07d-3e1baaf6bed8 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:18.223735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853c318-0e4b-45f8-bbb9-a8291fa8f085 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:18.262068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37ae97-a4c3-4a25-93e9-cf6e18f8c8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:18.262872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37ae97-a4c3-4a25-93e9-cf6e18f8c8ff None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:18.263152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37ae97-a4c3-4a25-93e9-cf6e18f8c8ff None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:18.263427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37ae97-a4c3-4a25-93e9-cf6e18f8c8ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:18.270994 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3a855-65fb-48de-9fce-34913d9aade4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:18.271943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3a855-65fb-48de-9fce-34913d9aade4 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:18.272442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3a855-65fb-48de-9fce-34913d9aade4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:18.272820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37ae97-a4c3-4a25-93e9-cf6e18f8c8ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:18.273208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3a855-65fb-48de-9fce-34913d9aade4 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:18.274171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3a855-65fb-48de-9fce-34913d9aade4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:18.274280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37ae97-a4c3-4a25-93e9-cf6e18f8c8ff None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:18.274790 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3a855-65fb-48de-9fce-34913d9aade4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:18.277613 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3a855-65fb-48de-9fce-34913d9aade4 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:18.287777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 423/857] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:33:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:18.294023 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3a855-65fb-48de-9fce-34913d9aade4 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-7c37ae97-a4c3-4a25-93e9-cf6e18f8c8ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:18.294169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:18.295028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:18.295912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:18.295912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:18.295912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:18.295912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:18.295912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:18.295912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:18.295912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:18.295912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:18.295912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 435/858] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:18.303375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37ae97-a4c3-4a25-93e9-cf6e18f8c8ff None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:18.322748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:18.342242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:18.425635 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37ae97-a4c3-4a25-93e9-cf6e18f8c8ff None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:18.428053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3a855-65fb-48de-9fce-34913d9aade4 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:18.462431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b391d0-e7f9-4085-b9db-f0f42106edbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:18.463246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b391d0-e7f9-4085-b9db-f0f42106edbe None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:18.463601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b391d0-e7f9-4085-b9db-f0f42106edbe None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:18.463945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b391d0-e7f9-4085-b9db-f0f42106edbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:18.471067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8c320-7435-4dc4-8c99-32be799dfaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:18.472458 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8c320-7435-4dc4-8c99-32be799dfaae None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:18.473414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8c320-7435-4dc4-8c99-32be799dfaae None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:18.473896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8c320-7435-4dc4-8c99-32be799dfaae None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:18.475267 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8c320-7435-4dc4-8c99-32be799dfaae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:18.475766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8c320-7435-4dc4-8c99-32be799dfaae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:18.477000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b391d0-e7f9-4085-b9db-f0f42106edbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:18.477901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8c320-7435-4dc4-8c99-32be799dfaae None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:18.478567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b391d0-e7f9-4085-b9db-f0f42106edbe None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:18.492433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 436/859] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:18.500681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8c320-7435-4dc4-8c99-32be799dfaae None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-15b391d0-e7f9-4085-b9db-f0f42106edbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:18.500826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:18.501654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:18.503054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:18.503054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:18.503054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:18.503054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:18.503054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 424/860] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:18.508938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b391d0-e7f9-4085-b9db-f0f42106edbe None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:18.599469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:18.606037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:18.617492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8c320-7435-4dc4-8c99-32be799dfaae None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:18.650848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b391d0-e7f9-4085-b9db-f0f42106edbe None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:18.657181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54e7ba-8120-490f-91f8-841e882ef062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:18.658107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54e7ba-8120-490f-91f8-841e882ef062 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:18.658653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54e7ba-8120-490f-91f8-841e882ef062 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:18.659177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54e7ba-8120-490f-91f8-841e882ef062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:18.660392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54e7ba-8120-490f-91f8-841e882ef062 None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=c151c53202f24faeb3accf802716652f) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:18.661019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54e7ba-8120-490f-91f8-841e882ef062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:18.662760 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54e7ba-8120-490f-91f8-841e882ef062 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:18.670149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 437/861] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:18 2022] GET /identity/v3/role_assignments?user.id=c151c53202f24faeb3accf802716652f&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:18.688012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe29234-b6e0-4bf0-80b3-45c574a7707f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:18.688734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe29234-b6e0-4bf0-80b3-45c574a7707f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:18.689024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe29234-b6e0-4bf0-80b3-45c574a7707f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:18.689301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe29234-b6e0-4bf0-80b3-45c574a7707f None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:18.690608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe29234-b6e0-4bf0-80b3-45c574a7707f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:18.690917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe29234-b6e0-4bf0-80b3-45c574a7707f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:18.692348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe29234-b6e0-4bf0-80b3-45c574a7707f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:18.708289 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 425/862] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:18.714168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54e7ba-8120-490f-91f8-841e882ef062 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:18.833227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54e7ba-8120-490f-91f8-841e882ef062 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:18.872108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970dc9ba-60d4-4d70-a6a2-2d7095698ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:18.872972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970dc9ba-60d4-4d70-a6a2-2d7095698ec4 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:18.873256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970dc9ba-60d4-4d70-a6a2-2d7095698ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:18.873532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970dc9ba-60d4-4d70-a6a2-2d7095698ec4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:18.882074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970dc9ba-60d4-4d70-a6a2-2d7095698ec4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:18.883509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970dc9ba-60d4-4d70-a6a2-2d7095698ec4 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-970dc9ba-60d4-4d70-a6a2-2d7095698ec4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:18.898282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:18.899214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:18.900145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:18.900145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:18.900145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:18.900145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:18.900145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:18.900145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:18.900145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:18.900145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:18.900145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 08:33:18.900145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:18.900145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 438/863] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:33:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:18.907690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe29234-b6e0-4bf0-80b3-45c574a7707f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:19.013200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe29234-b6e0-4bf0-80b3-45c574a7707f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:19.041866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:19.051011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e182489-32e3-4483-8e23-1e5f06139394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:19.051702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e182489-32e3-4483-8e23-1e5f06139394 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:19.052367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e182489-32e3-4483-8e23-1e5f06139394 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:19.053066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e182489-32e3-4483-8e23-1e5f06139394 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:19.053915 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e182489-32e3-4483-8e23-1e5f06139394 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:19.054225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e182489-32e3-4483-8e23-1e5f06139394 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:19.055658 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e182489-32e3-4483-8e23-1e5f06139394 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:19.074309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 426/864] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Jul 25 08:33:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:19.081691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970dc9ba-60d4-4d70-a6a2-2d7095698ec4 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:19.103003 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:19.193818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970dc9ba-60d4-4d70-a6a2-2d7095698ec4 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:19.230315 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a283a2-5d6b-474f-9494-872720578938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:19.231350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a283a2-5d6b-474f-9494-872720578938 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:19.231867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a283a2-5d6b-474f-9494-872720578938 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:19.232365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a283a2-5d6b-474f-9494-872720578938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:19.312595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a283a2-5d6b-474f-9494-872720578938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:19.312595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a283a2-5d6b-474f-9494-872720578938 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:19.327983 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:19.414017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 439/865] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:33:19.419520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e182489-32e3-4483-8e23-1e5f06139394 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:19.524425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e182489-32e3-4483-8e23-1e5f06139394 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:19.561921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec9307-ea69-4971-97b7-dc121b625ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:19.563114 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec9307-ea69-4971-97b7-dc121b625ee3 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:19.563597 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec9307-ea69-4971-97b7-dc121b625ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:19.564077 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec9307-ea69-4971-97b7-dc121b625ee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:19.575186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec9307-ea69-4971-97b7-dc121b625ee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:19.576921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec9307-ea69-4971-97b7-dc121b625ee3 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-54ec9307-ea69-4971-97b7-dc121b625ee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:19.595293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:19.596791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:19.597690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:19.597690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:19.597690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:19.597690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:19.599896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 427/866] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:19.605597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a283a2-5d6b-474f-9494-872720578938 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:19.612632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c98f8c-9096-45d5-b444-c3214a962def None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:19.612926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c98f8c-9096-45d5-b444-c3214a962def None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:19.613206 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c98f8c-9096-45d5-b444-c3214a962def None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:19.614808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 428/867] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:19.619524 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b705c6b-8a71-42ca-94ac-8802fc02e547 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:19.619813 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b705c6b-8a71-42ca-94ac-8802fc02e547 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:19.620094 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b705c6b-8a71-42ca-94ac-8802fc02e547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:19.638078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5b705c6b-8a71-42ca-94ac-8802fc02e547 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:19.639255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5b705c6b-8a71-42ca-94ac-8802fc02e547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:19.671565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:19.716900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 429/868] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:19.730325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a283a2-5d6b-474f-9494-872720578938 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:19.733196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b705c6b-8a71-42ca-94ac-8802fc02e547 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:19.765048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:19.771615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b95286-ab5c-4b91-9eae-bc02b6880835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:19.773155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b95286-ab5c-4b91-9eae-bc02b6880835 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:19.773155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b95286-ab5c-4b91-9eae-bc02b6880835 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:19.773155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b95286-ab5c-4b91-9eae-bc02b6880835 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:19.773828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b95286-ab5c-4b91-9eae-bc02b6880835 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:19.774136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b95286-ab5c-4b91-9eae-bc02b6880835 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:19.775505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b95286-ab5c-4b91-9eae-bc02b6880835 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:19.788608 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 440/869] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:19.793415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b95286-ab5c-4b91-9eae-bc02b6880835 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:19.862714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b705c6b-8a71-42ca-94ac-8802fc02e547 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:19.898846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7909f-4b5d-4eb5-8396-67cc23dccd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:19.899846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7909f-4b5d-4eb5-8396-67cc23dccd71 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:19.900373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7909f-4b5d-4eb5-8396-67cc23dccd71 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:19.900868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7909f-4b5d-4eb5-8396-67cc23dccd71 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:19.901785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7909f-4b5d-4eb5-8396-67cc23dccd71 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:19.903583 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7909f-4b5d-4eb5-8396-67cc23dccd71 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:19.904966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b95286-ab5c-4b91-9eae-bc02b6880835 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:19.933183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 430/870] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:33:19 2022] POST /identity/v3/services => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:19.952183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd6608-e082-419c-aba5-eecb34315037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:19.953197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd6608-e082-419c-aba5-eecb34315037 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:19.953547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd6608-e082-419c-aba5-eecb34315037 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:19.953892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd6608-e082-419c-aba5-eecb34315037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:19.963366 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd6608-e082-419c-aba5-eecb34315037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:19.965031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd6608-e082-419c-aba5-eecb34315037 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-99cd6608-e082-419c-aba5-eecb34315037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:19.982609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:19.983555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:19.984422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:19.984422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:19.984422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:19.984422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:19.984422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:19.984422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:19.984422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:19.984422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:19.984422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 441/871] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:19.990442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7909f-4b5d-4eb5-8396-67cc23dccd71 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:20.025792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:20.113840 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7909f-4b5d-4eb5-8396-67cc23dccd71 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:20.150318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c4b41-6e4c-4812-9ed9-5d6ce612e397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:20.151330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c4b41-6e4c-4812-9ed9-5d6ce612e397 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:20.151793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c4b41-6e4c-4812-9ed9-5d6ce612e397 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:20.152527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c4b41-6e4c-4812-9ed9-5d6ce612e397 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:20.153515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c4b41-6e4c-4812-9ed9-5d6ce612e397 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:20.154020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c4b41-6e4c-4812-9ed9-5d6ce612e397 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:20.155619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c4b41-6e4c-4812-9ed9-5d6ce612e397 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:20.162867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2408f-c867-4343-8cb1-3f1d1d363f58 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:20.163333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2408f-c867-4343-8cb1-3f1d1d363f58 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:20.163409 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2408f-c867-4343-8cb1-3f1d1d363f58 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:20.164968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 442/872] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:20.170075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbe3f3-404c-4731-8b26-5a10f2f654a7 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:20.170335 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbe3f3-404c-4731-8b26-5a10f2f654a7 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:20.170633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 431/873] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:20.170751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbe3f3-404c-4731-8b26-5a10f2f654a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:20.176102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c4b41-6e4c-4812-9ed9-5d6ce612e397 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:20.189464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-41dbe3f3-404c-4731-8b26-5a10f2f654a7 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:20.191393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-41dbe3f3-404c-4731-8b26-5a10f2f654a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:20.219636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:20.266683 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 443/874] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:20.283750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:20.292304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbe3f3-404c-4731-8b26-5a10f2f654a7 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:20.313693 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c4b41-6e4c-4812-9ed9-5d6ce612e397 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:20.367371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d0014-36d5-4997-8f4a-fcaf24549136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:20.368665 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d0014-36d5-4997-8f4a-fcaf24549136 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:20.369179 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d0014-36d5-4997-8f4a-fcaf24549136 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:20.370263 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d0014-36d5-4997-8f4a-fcaf24549136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:20.371588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d0014-36d5-4997-8f4a-fcaf24549136 None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=7d6be43abd85417c9fb7bbcb36a1f77d) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:20.372091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d0014-36d5-4997-8f4a-fcaf24549136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:20.375049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d0014-36d5-4997-8f4a-fcaf24549136 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:20.384279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 432/875] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:20 2022] GET /identity/v3/role_assignments?user.id=7d6be43abd85417c9fb7bbcb36a1f77d&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:20.412450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbe3f3-404c-4731-8b26-5a10f2f654a7 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:20.453972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489ecc3-fbc6-4d63-9aa2-e94663e99a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:20.454586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489ecc3-fbc6-4d63-9aa2-e94663e99a2e None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:20.454868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489ecc3-fbc6-4d63-9aa2-e94663e99a2e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:20.455145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489ecc3-fbc6-4d63-9aa2-e94663e99a2e None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:20.455943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489ecc3-fbc6-4d63-9aa2-e94663e99a2e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:20.457386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489ecc3-fbc6-4d63-9aa2-e94663e99a2e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:20.477211 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 444/876] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:33:20 2022] POST /identity/v3/services => generated 269 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:20.917522 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae21ef-f763-4e14-88b4-ff507fe1f726 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:20.917922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae21ef-f763-4e14-88b4-ff507fe1f726 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:20.918102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae21ef-f763-4e14-88b4-ff507fe1f726 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:20.919744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 433/877] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:20.925407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d7c53-1a47-413f-b101-97e579c944e3 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:20.925700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d7c53-1a47-413f-b101-97e579c944e3 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:20.925980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d7c53-1a47-413f-b101-97e579c944e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:20.944561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-5a4d7c53-1a47-413f-b101-97e579c944e3 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:20.945913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-5a4d7c53-1a47-413f-b101-97e579c944e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:20.963850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:21.023678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 445/878] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:21.046769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae21ef-f763-4e14-88b4-ff507fe1f726 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:21.156274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae21ef-f763-4e14-88b4-ff507fe1f726 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:21.197233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1917650-fd83-4924-a113-81b7c883a130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:21.198390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1917650-fd83-4924-a113-81b7c883a130 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:21.198890 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1917650-fd83-4924-a113-81b7c883a130 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:21.199366 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1917650-fd83-4924-a113-81b7c883a130 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:21.210667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1917650-fd83-4924-a113-81b7c883a130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-d1917650-fd83-4924-a113-81b7c883a130 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:21.219189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:21.220127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:21.221032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:21.221032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:21.221032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:21.221032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:21.221032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:21.221032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:21.221032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:21.221032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:21.221032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:21.221032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:21.221032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 08:33:21.221032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:21.223786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 434/879] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Jul 25 08:33:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:21.229438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d7c53-1a47-413f-b101-97e579c944e3 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:21.330106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d7c53-1a47-413f-b101-97e579c944e3 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:21.364946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec465d-5ceb-4cb1-83f9-3d44fa6ba2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:21.365614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec465d-5ceb-4cb1-83f9-3d44fa6ba2cf None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:21.365896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec465d-5ceb-4cb1-83f9-3d44fa6ba2cf None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:21.366172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec465d-5ceb-4cb1-83f9-3d44fa6ba2cf None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:21.366913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec465d-5ceb-4cb1-83f9-3d44fa6ba2cf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:21.367218 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec465d-5ceb-4cb1-83f9-3d44fa6ba2cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:21.368588 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec465d-5ceb-4cb1-83f9-3d44fa6ba2cf None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:21.376890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 446/880] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Jul 25 08:33:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:21.382274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1917650-fd83-4924-a113-81b7c883a130 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:21.398308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:21.491776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1917650-fd83-4924-a113-81b7c883a130 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:21.526645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b9259-4bf5-4f3f-886d-b02afb21716b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:21.527716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b9259-4bf5-4f3f-886d-b02afb21716b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:21.528002 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b9259-4bf5-4f3f-886d-b02afb21716b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:21.528334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b9259-4bf5-4f3f-886d-b02afb21716b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:21.539434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b9259-4bf5-4f3f-886d-b02afb21716b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:21.540907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b9259-4bf5-4f3f-886d-b02afb21716b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-908b9259-4bf5-4f3f-886d-b02afb21716b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:21.558259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:21.559182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:21.560066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:21.560066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:21.560066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:21.560066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:21.560066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 435/881] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:21.567079 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec465d-5ceb-4cb1-83f9-3d44fa6ba2cf None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:21.581269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:21.682177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec465d-5ceb-4cb1-83f9-3d44fa6ba2cf None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:21.717214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b578f-6400-4602-ae7d-58a6e1d43209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:21.717879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b578f-6400-4602-ae7d-58a6e1d43209 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:21.718164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b578f-6400-4602-ae7d-58a6e1d43209 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:21.718438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b578f-6400-4602-ae7d-58a6e1d43209 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:21.719156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b578f-6400-4602-ae7d-58a6e1d43209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:21.719459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b578f-6400-4602-ae7d-58a6e1d43209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:21.720874 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b578f-6400-4602-ae7d-58a6e1d43209 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:21.733024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 447/882] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:21.738232 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b9259-4bf5-4f3f-886d-b02afb21716b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:21.804711 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:21.843535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b9259-4bf5-4f3f-886d-b02afb21716b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:21.878229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eab6af-b869-4ff5-a4ed-0ddb6a2ee0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:21.879054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eab6af-b869-4ff5-a4ed-0ddb6a2ee0b7 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:21.879341 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eab6af-b869-4ff5-a4ed-0ddb6a2ee0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:21.879619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eab6af-b869-4ff5-a4ed-0ddb6a2ee0b7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:21.887407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eab6af-b869-4ff5-a4ed-0ddb6a2ee0b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:21.888865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eab6af-b869-4ff5-a4ed-0ddb6a2ee0b7 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-34eab6af-b869-4ff5-a4ed-0ddb6a2ee0b7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:21.904269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:21.905238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:21.906145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:21.906145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:21.906145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:21.906145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:21.906145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:21.906145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:21.906145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:21.906145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:21.906145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 08:33:21.906145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:21.906145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 436/883] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Jul 25 08:33:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:21.911248 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b578f-6400-4602-ae7d-58a6e1d43209 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:21.982936 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:22.015349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b578f-6400-4602-ae7d-58a6e1d43209 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:22.050915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47a864-b7fb-46f2-9629-58766ec0e0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:22.051594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47a864-b7fb-46f2-9629-58766ec0e0e6 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.051879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47a864-b7fb-46f2-9629-58766ec0e0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.052157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47a864-b7fb-46f2-9629-58766ec0e0e6 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.053031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47a864-b7fb-46f2-9629-58766ec0e0e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:22.053413 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47a864-b7fb-46f2-9629-58766ec0e0e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:22.054976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47a864-b7fb-46f2-9629-58766ec0e0e6 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:22.073638 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 448/884] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Jul 25 08:33:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:22.078965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eab6af-b869-4ff5-a4ed-0ddb6a2ee0b7 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:22.109463 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fdd82-eb40-4ebc-9324-47a7225d88a9 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.109768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fdd82-eb40-4ebc-9324-47a7225d88a9 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.110031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fdd82-eb40-4ebc-9324-47a7225d88a9 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.112206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 449/885] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:22.117851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc41c4b-ca0f-4bf7-a809-bfe74ac9d51a None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.118150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc41c4b-ca0f-4bf7-a809-bfe74ac9d51a None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.118438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc41c4b-ca0f-4bf7-a809-bfe74ac9d51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.138710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-5cc41c4b-ca0f-4bf7-a809-bfe74ac9d51a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:22.140047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-5cc41c4b-ca0f-4bf7-a809-bfe74ac9d51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:22.188562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eab6af-b869-4ff5-a4ed-0ddb6a2ee0b7 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:22.217295 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 450/886] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:22.217945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:22.229641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a8e2e-6992-4649-987c-4f1e202222f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:22.230305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a8e2e-6992-4649-987c-4f1e202222f5 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.230595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a8e2e-6992-4649-987c-4f1e202222f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.230878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a8e2e-6992-4649-987c-4f1e202222f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.240192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc41c4b-ca0f-4bf7-a809-bfe74ac9d51a None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:22.322961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:22.324533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a8e2e-6992-4649-987c-4f1e202222f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:22.326013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a8e2e-6992-4649-987c-4f1e202222f5 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:22.370592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc41c4b-ca0f-4bf7-a809-bfe74ac9d51a None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:22.407921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f496d1-3f19-4e5a-b215-30874b6c3328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:22.409698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f496d1-3f19-4e5a-b215-30874b6c3328 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.410271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f496d1-3f19-4e5a-b215-30874b6c3328 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.410776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f496d1-3f19-4e5a-b215-30874b6c3328 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.411636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f496d1-3f19-4e5a-b215-30874b6c3328 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:22.413434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f496d1-3f19-4e5a-b215-30874b6c3328 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-10f496d1-3f19-4e5a-b215-30874b6c3328 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:22.427206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:22.428217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 08:33:22.429080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:22.437417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 437/887] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:22 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:33:22.437585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 451/888] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:33:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:22.440088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a8e2e-6992-4649-987c-4f1e202222f5 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:22.441535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f496d1-3f19-4e5a-b215-30874b6c3328 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:22.465389 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:22.548533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a8e2e-6992-4649-987c-4f1e202222f5 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:22.558630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f496d1-3f19-4e5a-b215-30874b6c3328 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:22.582224 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:22.583065 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a19d8-643d-4bb4-96cf-c7570ef5241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:22.583887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a19d8-643d-4bb4-96cf-c7570ef5241d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.584171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a19d8-643d-4bb4-96cf-c7570ef5241d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.584447 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a19d8-643d-4bb4-96cf-c7570ef5241d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.594468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54e60f-3877-4073-b0fa-eaa5bb8bd87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:22.594890 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a19d8-643d-4bb4-96cf-c7570ef5241d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:22.595148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54e60f-3877-4073-b0fa-eaa5bb8bd87b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.595426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54e60f-3877-4073-b0fa-eaa5bb8bd87b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.595705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54e60f-3877-4073-b0fa-eaa5bb8bd87b None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.596900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a19d8-643d-4bb4-96cf-c7570ef5241d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:22.596998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54e60f-3877-4073-b0fa-eaa5bb8bd87b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:22.597268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54e60f-3877-4073-b0fa-eaa5bb8bd87b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:22.598633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54e60f-3877-4073-b0fa-eaa5bb8bd87b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:22.603404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 452/889] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:33:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:22.604466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80236e5-01f1-46bb-bf07-8158d65a790c None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.604786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80236e5-01f1-46bb-bf07-8158d65a790c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.605067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80236e5-01f1-46bb-bf07-8158d65a790c None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.606439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 453/890] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:22.607378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369aab7f-3947-4fd3-8325-ac4686537ccf None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.607668 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369aab7f-3947-4fd3-8325-ac4686537ccf None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.608033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369aab7f-3947-4fd3-8325-ac4686537ccf None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.609196 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 454/891] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:22.612833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369aab7f-3947-4fd3-8325-ac4686537ccf None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-186a19d8-643d-4bb4-96cf-c7570ef5241d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:22.620894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:22.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:22.623277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:22.623277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:22.623277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:22.623277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:22.623277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 438/892] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:22.623622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed145aae-9e9c-44cc-8d8b-02a0213101b3 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.623833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed145aae-9e9c-44cc-8d8b-02a0213101b3 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.624114 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed145aae-9e9c-44cc-8d8b-02a0213101b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.642224 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ed145aae-9e9c-44cc-8d8b-02a0213101b3 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:22.643891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ed145aae-9e9c-44cc-8d8b-02a0213101b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:22.709350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 439/893] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:22.710731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a683efc-f815-449d-a18d-da2f2a4038da None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.710974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a683efc-f815-449d-a18d-da2f2a4038da None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.711258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a683efc-f815-449d-a18d-da2f2a4038da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.716075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369aab7f-3947-4fd3-8325-ac4686537ccf None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:22.724009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:22.729904 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5a683efc-f815-449d-a18d-da2f2a4038da None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:22.731373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5a683efc-f815-449d-a18d-da2f2a4038da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:22.753496 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d361a-9f9d-4892-8b45-c7fa0eb86538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:22.754171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d361a-9f9d-4892-8b45-c7fa0eb86538 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.754459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d361a-9f9d-4892-8b45-c7fa0eb86538 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.754739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d361a-9f9d-4892-8b45-c7fa0eb86538 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.755510 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d361a-9f9d-4892-8b45-c7fa0eb86538 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:22.755841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d361a-9f9d-4892-8b45-c7fa0eb86538 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:22.757247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d361a-9f9d-4892-8b45-c7fa0eb86538 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:22.762404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 455/894] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:33:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:22.763158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d361a-9f9d-4892-8b45-c7fa0eb86538 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:22.801229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 440/895] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:22.802310 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a683efc-f815-449d-a18d-da2f2a4038da None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:22.812459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:22.867345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d361a-9f9d-4892-8b45-c7fa0eb86538 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:22.899940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a683efc-f815-449d-a18d-da2f2a4038da None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:22.900936 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0dbbc-d782-4955-9588-24174e488631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:22.901606 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0dbbc-d782-4955-9588-24174e488631 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.902150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0dbbc-d782-4955-9588-24174e488631 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.902437 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0dbbc-d782-4955-9588-24174e488631 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.903165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0dbbc-d782-4955-9588-24174e488631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:22.903471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0dbbc-d782-4955-9588-24174e488631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:22.904868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0dbbc-d782-4955-9588-24174e488631 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:22.916590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 456/896] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:22.917396 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0dbbc-d782-4955-9588-24174e488631 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:22.934434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0352742-7237-412c-9baa-d628538de913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:22.935238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0352742-7237-412c-9baa-d628538de913 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:22.935520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0352742-7237-412c-9baa-d628538de913 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:22.935795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0352742-7237-412c-9baa-d628538de913 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:22.936445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0352742-7237-412c-9baa-d628538de913 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:22.937834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0352742-7237-412c-9baa-d628538de913 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:22.940685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-f0352742-7237-412c-9baa-d628538de913 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:22.952117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:22.953269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:22.954084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 441/897] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Jul 25 08:33:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:22.955611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0352742-7237-412c-9baa-d628538de913 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:23.024328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0dbbc-d782-4955-9588-24174e488631 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:23.035351 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:23.054718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0352742-7237-412c-9baa-d628538de913 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:23.059208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6f21-d08f-4858-b9cc-41586aa74e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:23.059871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa6f21-d08f-4858-b9cc-41586aa74e89 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:23.060271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa6f21-d08f-4858-b9cc-41586aa74e89 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:23.060550 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa6f21-d08f-4858-b9cc-41586aa74e89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:23.061380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa6f21-d08f-4858-b9cc-41586aa74e89 None admin] RBAC: Adding query filter params (interface=public, service_id=1fa51f14df24420cacd24017dc0c2a21, region_id=RegionOne) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:23.061700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa6f21-d08f-4858-b9cc-41586aa74e89 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:23.063199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa6f21-d08f-4858-b9cc-41586aa74e89 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:23.068933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 457/898] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Jul 25 08:33:22 2022] GET /identity/v3/endpoints?service_id=1fa51f14df24420cacd24017dc0c2a21&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:23.069703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6f21-d08f-4858-b9cc-41586aa74e89 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:23.088795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9925ea-8371-4317-882f-663b6213c8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:23.089602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9925ea-8371-4317-882f-663b6213c8b6 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:23.089879 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9925ea-8371-4317-882f-663b6213c8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:23.090175 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9925ea-8371-4317-882f-663b6213c8b6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:23.090829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9925ea-8371-4317-882f-663b6213c8b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:23.092172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9925ea-8371-4317-882f-663b6213c8b6 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-5a9925ea-8371-4317-882f-663b6213c8b6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:23.104229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:23.105164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:23.105995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 442/899] 10.222.0.52 () {56 vars in 1132 bytes} [Mon Jul 25 08:33:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:23.106833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9925ea-8371-4317-882f-663b6213c8b6 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:23.167471 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:23.169716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6f21-d08f-4858-b9cc-41586aa74e89 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:23.203699 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec5835-4577-494e-8092-fffc7ca44320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:23.204485 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec5835-4577-494e-8092-fffc7ca44320 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:23.204793 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec5835-4577-494e-8092-fffc7ca44320 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:23.205131 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec5835-4577-494e-8092-fffc7ca44320 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:23.205832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9925ea-8371-4317-882f-663b6213c8b6 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:23.212684 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec5835-4577-494e-8092-fffc7ca44320 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:23.214113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec5835-4577-494e-8092-fffc7ca44320 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-37ec5835-4577-494e-8092-fffc7ca44320 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:23.228019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:23.229098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:23.230172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:23.230172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:23.230172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:23.230172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:23.230172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:23.230172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:23.230172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:23.230172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:23.230172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 458/900] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:23.230172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec5835-4577-494e-8092-fffc7ca44320 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:23.240797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a0335-ba74-47bc-8c30-935fe294f1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:23.241463 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a0335-ba74-47bc-8c30-935fe294f1c8 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:23.241751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a0335-ba74-47bc-8c30-935fe294f1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:23.242026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a0335-ba74-47bc-8c30-935fe294f1c8 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:23.242747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a0335-ba74-47bc-8c30-935fe294f1c8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:23.243058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a0335-ba74-47bc-8c30-935fe294f1c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:23.244420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a0335-ba74-47bc-8c30-935fe294f1c8 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:23.248988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 443/901] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Jul 25 08:33:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:23.249750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a0335-ba74-47bc-8c30-935fe294f1c8 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:23.266419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:23.334320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec5835-4577-494e-8092-fffc7ca44320 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:23.350015 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a0335-ba74-47bc-8c30-935fe294f1c8 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:23.367391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d26192-5612-4671-a3b9-4fc87d0f8c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:23.368056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d26192-5612-4671-a3b9-4fc87d0f8c59 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:23.368342 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d26192-5612-4671-a3b9-4fc87d0f8c59 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:23.368651 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d26192-5612-4671-a3b9-4fc87d0f8c59 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:23.369351 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d26192-5612-4671-a3b9-4fc87d0f8c59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:23.370719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d26192-5612-4671-a3b9-4fc87d0f8c59 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:23.374838 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:23.376633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 459/902] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Jul 25 08:33:23 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:33:23.377890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d26192-5612-4671-a3b9-4fc87d0f8c59 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:23.390413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d3041-14c0-4a70-8beb-d498ab666b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:23.391567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d3041-14c0-4a70-8beb-d498ab666b41 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:23.391567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d3041-14c0-4a70-8beb-d498ab666b41 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:23.391798 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d3041-14c0-4a70-8beb-d498ab666b41 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:23.392393 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d3041-14c0-4a70-8beb-d498ab666b41 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:23.393305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d3041-14c0-4a70-8beb-d498ab666b41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:23.394883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d3041-14c0-4a70-8beb-d498ab666b41 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:23.400383 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 444/903] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Jul 25 08:33:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:23.401085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d3041-14c0-4a70-8beb-d498ab666b41 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:23.500641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:23.505875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d26192-5612-4671-a3b9-4fc87d0f8c59 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:23.516928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d3041-14c0-4a70-8beb-d498ab666b41 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:23.543799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a9024-aea7-4e02-8c0d-8020dfd9f979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:23.544692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a9024-aea7-4e02-8c0d-8020dfd9f979 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:23.544982 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a9024-aea7-4e02-8c0d-8020dfd9f979 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:23.545316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a9024-aea7-4e02-8c0d-8020dfd9f979 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:23.546112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a9024-aea7-4e02-8c0d-8020dfd9f979 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:23.546421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a9024-aea7-4e02-8c0d-8020dfd9f979 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:23.547839 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a9024-aea7-4e02-8c0d-8020dfd9f979 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:23.559278 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b24a57-58ca-4862-b0d4-cd88e9781602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:23.559958 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b24a57-58ca-4862-b0d4-cd88e9781602 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:23.560245 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b24a57-58ca-4862-b0d4-cd88e9781602 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:23.560524 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b24a57-58ca-4862-b0d4-cd88e9781602 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:23.561690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 460/904] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:23.561812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b24a57-58ca-4862-b0d4-cd88e9781602 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:23.561898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a9024-aea7-4e02-8c0d-8020dfd9f979 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:23.561970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b24a57-58ca-4862-b0d4-cd88e9781602 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:23.563332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b24a57-58ca-4862-b0d4-cd88e9781602 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:23.569455 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 445/905] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Jul 25 08:33:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:23.570280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b24a57-58ca-4862-b0d4-cd88e9781602 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:23.648588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:23.677118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a9024-aea7-4e02-8c0d-8020dfd9f979 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:23.712632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b24a57-58ca-4862-b0d4-cd88e9781602 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:23.716314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d77e50-bf22-46e7-9b77-9d5685116fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:23.717112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d77e50-bf22-46e7-9b77-9d5685116fd5 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:23.717469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d77e50-bf22-46e7-9b77-9d5685116fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:23.717824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d77e50-bf22-46e7-9b77-9d5685116fd5 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:23.718626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d77e50-bf22-46e7-9b77-9d5685116fd5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:23.719007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d77e50-bf22-46e7-9b77-9d5685116fd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:23.720633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d77e50-bf22-46e7-9b77-9d5685116fd5 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:23.726983 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 461/906] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Jul 25 08:33:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:23.737739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d77e50-bf22-46e7-9b77-9d5685116fd5 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:23.751565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734000f-3717-4f31-8c54-e74fbbbb1a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:23.753206 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734000f-3717-4f31-8c54-e74fbbbb1a2a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:23.753687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734000f-3717-4f31-8c54-e74fbbbb1a2a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:23.754103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734000f-3717-4f31-8c54-e74fbbbb1a2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:23.755125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734000f-3717-4f31-8c54-e74fbbbb1a2a None admin] RBAC: Adding query filter params (role.id=1ac8ff72ed404116b52f57f20668457d, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=c151c53202f24faeb3accf802716652f) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:23.755672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734000f-3717-4f31-8c54-e74fbbbb1a2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:23.757374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734000f-3717-4f31-8c54-e74fbbbb1a2a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:23.763636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 446/907] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:23 2022] GET /identity/v3/role_assignments?user.id=c151c53202f24faeb3accf802716652f&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=1ac8ff72ed404116b52f57f20668457d => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:23.766427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:23.856419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d77e50-bf22-46e7-9b77-9d5685116fd5 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:23.894341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9304b27-2203-402b-9cfa-00701e6a87c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:23.895412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9304b27-2203-402b-9cfa-00701e6a87c9 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:23.895896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9304b27-2203-402b-9cfa-00701e6a87c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:23.896365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9304b27-2203-402b-9cfa-00701e6a87c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:23.897434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9304b27-2203-402b-9cfa-00701e6a87c9 None admin] RBAC: Adding query filter params (interface=public, service_id=9d95cb32e3374dd1befe15f298fee42c, region_id=RegionOne) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:23.897951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9304b27-2203-402b-9cfa-00701e6a87c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:23.899617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9304b27-2203-402b-9cfa-00701e6a87c9 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:23.906415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 462/908] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Jul 25 08:33:23 2022] GET /identity/v3/endpoints?service_id=9d95cb32e3374dd1befe15f298fee42c&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:23.912630 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734000f-3717-4f31-8c54-e74fbbbb1a2a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:24.015856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734000f-3717-4f31-8c54-e74fbbbb1a2a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:24.036902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:24.052081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef24fd1-1427-4c46-bda3-3ee90ce33068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:24.052786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef24fd1-1427-4c46-bda3-3ee90ce33068 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:24.053070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef24fd1-1427-4c46-bda3-3ee90ce33068 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:24.053347 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef24fd1-1427-4c46-bda3-3ee90ce33068 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:24.054036 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef24fd1-1427-4c46-bda3-3ee90ce33068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:24.055404 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef24fd1-1427-4c46-bda3-3ee90ce33068 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:24.061731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 447/909] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Jul 25 08:33:23 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:33:25.412539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7449eb5-1f64-4310-81c1-1ad1188036f5 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:25.413069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7449eb5-1f64-4310-81c1-1ad1188036f5 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:25.413179 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7449eb5-1f64-4310-81c1-1ad1188036f5 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:25.414757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 463/910] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:25.420160 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4709d-6918-4a08-a356-3cc955bd6fdc None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:25.420549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4709d-6918-4a08-a356-3cc955bd6fdc None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:25.420758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4709d-6918-4a08-a356-3cc955bd6fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:25.439637 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-f5f4709d-6918-4a08-a356-3cc955bd6fdc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:25.440858 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f5f4709d-6918-4a08-a356-3cc955bd6fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:25.510428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 448/911] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:25.532515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7449eb5-1f64-4310-81c1-1ad1188036f5 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:25.624772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:25.641728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7449eb5-1f64-4310-81c1-1ad1188036f5 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:25.662473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dcf34-2c5f-410c-ae42-4f0cfd569223 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:25.663116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dcf34-2c5f-410c-ae42-4f0cfd569223 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:25.663595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dcf34-2c5f-410c-ae42-4f0cfd569223 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:25.665539 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 449/912] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:25.671143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f823713-77d6-4aa3-a38b-3a6f21283b6f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:25.671695 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f823713-77d6-4aa3-a38b-3a6f21283b6f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:25.672170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f823713-77d6-4aa3-a38b-3a6f21283b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:25.678512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952b84f-90c8-462d-acd5-b187ea51198d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:25.679320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952b84f-90c8-462d-acd5-b187ea51198d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:25.679609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952b84f-90c8-462d-acd5-b187ea51198d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:25.679885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952b84f-90c8-462d-acd5-b187ea51198d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:25.680546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952b84f-90c8-462d-acd5-b187ea51198d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:25.681945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952b84f-90c8-462d-acd5-b187ea51198d None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:25.692346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-8f823713-77d6-4aa3-a38b-3a6f21283b6f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:25.693932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8f823713-77d6-4aa3-a38b-3a6f21283b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-3952b84f-90c8-462d-acd5-b187ea51198d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:25.694203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:25.695128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:25.695966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 464/913] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:33:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:25.701649 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952b84f-90c8-462d-acd5-b187ea51198d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:25.792420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 450/914] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:25.810796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983744fb-fafb-4a6d-80b5-a04d17ed4f3d None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:25.811082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983744fb-fafb-4a6d-80b5-a04d17ed4f3d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:25.811369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983744fb-fafb-4a6d-80b5-a04d17ed4f3d None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:25.813383 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 451/915] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:25.814081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983744fb-fafb-4a6d-80b5-a04d17ed4f3d None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:25.815616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952b84f-90c8-462d-acd5-b187ea51198d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:25.851802 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590077c-e111-47da-956d-00f91ca80b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:25.852563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590077c-e111-47da-956d-00f91ca80b48 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:25.852879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590077c-e111-47da-956d-00f91ca80b48 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:25.853157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590077c-e111-47da-956d-00f91ca80b48 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:25.853875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590077c-e111-47da-956d-00f91ca80b48 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:25.854180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590077c-e111-47da-956d-00f91ca80b48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:25.855547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590077c-e111-47da-956d-00f91ca80b48 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:25.860426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 465/916] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:33:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:25.860426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112e825-5eac-418a-b032-3b5f9fefe616 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:25.860426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112e825-5eac-418a-b032-3b5f9fefe616 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:25.860426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112e825-5eac-418a-b032-3b5f9fefe616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:25.870047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:25.880624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4112e825-5eac-418a-b032-3b5f9fefe616 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:25.881768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4112e825-5eac-418a-b032-3b5f9fefe616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:25.892005 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:25.930153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983744fb-fafb-4a6d-80b5-a04d17ed4f3d None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:25.954486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 466/917] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:25.955771 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112e825-5eac-418a-b032-3b5f9fefe616 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:25.976480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46079712-0ccc-46ca-87ea-ca24a6676a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:25.977195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46079712-0ccc-46ca-87ea-ca24a6676a06 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:25.977484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46079712-0ccc-46ca-87ea-ca24a6676a06 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:25.978209 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46079712-0ccc-46ca-87ea-ca24a6676a06 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:25.978954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46079712-0ccc-46ca-87ea-ca24a6676a06 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:25.980341 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46079712-0ccc-46ca-87ea-ca24a6676a06 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:25.995203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 452/918] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:33:25 2022] POST /identity/v3/services => generated 241 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:25.996014 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46079712-0ccc-46ca-87ea-ca24a6676a06 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:26.077876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112e825-5eac-418a-b032-3b5f9fefe616 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:26.124714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:26.128303 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b40749-00eb-48e2-b99b-590a3cd58052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:26.129018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b40749-00eb-48e2-b99b-590a3cd58052 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:26.129293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b40749-00eb-48e2-b99b-590a3cd58052 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:26.129572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b40749-00eb-48e2-b99b-590a3cd58052 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:26.129747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46079712-0ccc-46ca-87ea-ca24a6676a06 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:26.130310 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b40749-00eb-48e2-b99b-590a3cd58052 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:26.130621 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b40749-00eb-48e2-b99b-590a3cd58052 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:26.132385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b40749-00eb-48e2-b99b-590a3cd58052 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:26.145816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 467/919] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Jul 25 08:33:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:26.145816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabf096-9e77-4fd9-a4f1-62705f6784c1 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:26.145816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabf096-9e77-4fd9-a4f1-62705f6784c1 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:26.145816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabf096-9e77-4fd9-a4f1-62705f6784c1 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:26.145816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 468/920] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:26.148008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04116cff-6583-44cc-bb7a-92708bc8cd3c None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:26.148308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04116cff-6583-44cc-bb7a-92708bc8cd3c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:26.148618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04116cff-6583-44cc-bb7a-92708bc8cd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:26.156311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:26.168058 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-04116cff-6583-44cc-bb7a-92708bc8cd3c None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:26.169353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-04116cff-6583-44cc-bb7a-92708bc8cd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:26.181737 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3ca2f-588e-4129-84a5-194dd1e37712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:26.182561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3ca2f-588e-4129-84a5-194dd1e37712 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:26.182847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3ca2f-588e-4129-84a5-194dd1e37712 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:26.183118 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3ca2f-588e-4129-84a5-194dd1e37712 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:26.195162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3ca2f-588e-4129-84a5-194dd1e37712 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-d0d3ca2f-588e-4129-84a5-194dd1e37712 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:26.203930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:26.204987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:26.206353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 453/921] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Jul 25 08:33:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:26.207066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3ca2f-588e-4129-84a5-194dd1e37712 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:26.255319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 469/922] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:26.256147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04116cff-6583-44cc-bb7a-92708bc8cd3c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:26.328511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3ca2f-588e-4129-84a5-194dd1e37712 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:26.363970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe1d30-ccb9-47f9-aed1-6ff3b56d3796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:26.364672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe1d30-ccb9-47f9-aed1-6ff3b56d3796 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:26.364955 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe1d30-ccb9-47f9-aed1-6ff3b56d3796 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:26.365231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe1d30-ccb9-47f9-aed1-6ff3b56d3796 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:26.365973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe1d30-ccb9-47f9-aed1-6ff3b56d3796 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:26.367336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe1d30-ccb9-47f9-aed1-6ff3b56d3796 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:26.367502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04116cff-6583-44cc-bb7a-92708bc8cd3c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:26.400609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 454/923] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:33:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:26.400609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe1d30-ccb9-47f9-aed1-6ff3b56d3796 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:26.418911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:26.421107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:26.432838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f01eb-f4a8-434a-b494-d7533c17d017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:26.434053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f01eb-f4a8-434a-b494-d7533c17d017 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:26.434617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f01eb-f4a8-434a-b494-d7533c17d017 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:26.434900 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f01eb-f4a8-434a-b494-d7533c17d017 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:26.436037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f01eb-f4a8-434a-b494-d7533c17d017 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:26.436404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f01eb-f4a8-434a-b494-d7533c17d017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:26.438716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f01eb-f4a8-434a-b494-d7533c17d017 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:26.451713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 470/924] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Jul 25 08:33:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:26.457633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f01eb-f4a8-434a-b494-d7533c17d017 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:26.527771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe1d30-ccb9-47f9-aed1-6ff3b56d3796 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:26.567942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44d85a-dbfb-4c03-927a-f01e713af45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:26.568942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44d85a-dbfb-4c03-927a-f01e713af45d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:26.569291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44d85a-dbfb-4c03-927a-f01e713af45d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:26.569634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44d85a-dbfb-4c03-927a-f01e713af45d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:26.570475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44d85a-dbfb-4c03-927a-f01e713af45d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:26.572831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44d85a-dbfb-4c03-927a-f01e713af45d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-fa44d85a-dbfb-4c03-927a-f01e713af45d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:26.587587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:26.588599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:26.589495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 455/925] 10.222.0.52 () {56 vars in 1132 bytes} [Mon Jul 25 08:33:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:26.594739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44d85a-dbfb-4c03-927a-f01e713af45d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:26.610477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f01eb-f4a8-434a-b494-d7533c17d017 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:26.652610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0d41d-cb13-4200-9d07-060cc557afd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:26.653336 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0d41d-cb13-4200-9d07-060cc557afd7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:26.653646 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0d41d-cb13-4200-9d07-060cc557afd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:26.653902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0d41d-cb13-4200-9d07-060cc557afd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:26.666230 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0d41d-cb13-4200-9d07-060cc557afd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:26.667750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0d41d-cb13-4200-9d07-060cc557afd7 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-2cb0d41d-cb13-4200-9d07-060cc557afd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:26.691095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:26.692026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:26.692976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:26.692976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:26.692976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:26.692976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:26.692976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 471/926] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:26.696865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:26.697965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0d41d-cb13-4200-9d07-060cc557afd7 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:26.725169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44d85a-dbfb-4c03-927a-f01e713af45d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:26.754233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:26.763661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328320e4-d912-4600-89bb-26c6e759aa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:26.764544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328320e4-d912-4600-89bb-26c6e759aa53 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:26.764995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328320e4-d912-4600-89bb-26c6e759aa53 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:26.765516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328320e4-d912-4600-89bb-26c6e759aa53 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:26.766519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328320e4-d912-4600-89bb-26c6e759aa53 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:26.766970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328320e4-d912-4600-89bb-26c6e759aa53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:26.768656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328320e4-d912-4600-89bb-26c6e759aa53 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:26.774265 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 456/927] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Jul 25 08:33:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:26.780228 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328320e4-d912-4600-89bb-26c6e759aa53 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:26.827991 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0d41d-cb13-4200-9d07-060cc557afd7 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:26.867486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c2c04-3c74-48c9-a150-a0f6df70f2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:26.868395 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c2c04-3c74-48c9-a150-a0f6df70f2ac None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:26.868851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c2c04-3c74-48c9-a150-a0f6df70f2ac None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:26.869271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c2c04-3c74-48c9-a150-a0f6df70f2ac None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:26.870165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c2c04-3c74-48c9-a150-a0f6df70f2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:26.870615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c2c04-3c74-48c9-a150-a0f6df70f2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:26.872129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c2c04-3c74-48c9-a150-a0f6df70f2ac None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:26.885450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 472/928] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:26.890956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c2c04-3c74-48c9-a150-a0f6df70f2ac None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:26.902238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328320e4-d912-4600-89bb-26c6e759aa53 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:26.937234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d98550-586d-474f-846f-8d4d3e946a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:26.938227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d98550-586d-474f-846f-8d4d3e946a65 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:26.938709 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d98550-586d-474f-846f-8d4d3e946a65 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:26.939181 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d98550-586d-474f-846f-8d4d3e946a65 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:26.940137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d98550-586d-474f-846f-8d4d3e946a65 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:26.940673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d98550-586d-474f-846f-8d4d3e946a65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:26.942440 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d98550-586d-474f-846f-8d4d3e946a65 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:26.952703 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 457/929] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Jul 25 08:33:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:26.954213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d98550-586d-474f-846f-8d4d3e946a65 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:26.972942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:27.016147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c2c04-3c74-48c9-a150-a0f6df70f2ac None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:27.019880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:27.054951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b987c-4043-447f-b068-1a1ff3e1f292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:27.056114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b987c-4043-447f-b068-1a1ff3e1f292 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:27.056721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b987c-4043-447f-b068-1a1ff3e1f292 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:27.057283 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b987c-4043-447f-b068-1a1ff3e1f292 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:27.065386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b987c-4043-447f-b068-1a1ff3e1f292 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:27.067164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b987c-4043-447f-b068-1a1ff3e1f292 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:27.082094 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d98550-586d-474f-846f-8d4d3e946a65 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-b81b987c-4043-447f-b068-1a1ff3e1f292 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:27.083953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:27.085080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:27.085997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:27.085997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:27.085997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:27.085997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:27.085997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:27.085997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:27.085997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:27.085997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:27.085997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 08:33:27.085997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:27.088407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 473/930] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Jul 25 08:33:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:27.094447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b987c-4043-447f-b068-1a1ff3e1f292 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:27.123996 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d139ea6-46b2-4bbe-a46d-e641a952d200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:27.124690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d139ea6-46b2-4bbe-a46d-e641a952d200 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:27.124977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d139ea6-46b2-4bbe-a46d-e641a952d200 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:27.125256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d139ea6-46b2-4bbe-a46d-e641a952d200 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:27.126016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d139ea6-46b2-4bbe-a46d-e641a952d200 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:27.127422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d139ea6-46b2-4bbe-a46d-e641a952d200 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:27.157227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 458/931] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:33:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:27.230325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b987c-4043-447f-b068-1a1ff3e1f292 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:27.275497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0bc65-1590-4e1f-b3c6-28abcbd05a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:27.276184 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0bc65-1590-4e1f-b3c6-28abcbd05a63 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:27.276470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0bc65-1590-4e1f-b3c6-28abcbd05a63 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:27.277031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0bc65-1590-4e1f-b3c6-28abcbd05a63 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:27.277892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0bc65-1590-4e1f-b3c6-28abcbd05a63 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:27.278203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0bc65-1590-4e1f-b3c6-28abcbd05a63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:27.279663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0bc65-1590-4e1f-b3c6-28abcbd05a63 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:27.306788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 474/932] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Jul 25 08:33:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:27.312213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d139ea6-46b2-4bbe-a46d-e641a952d200 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:27.381620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:27.423068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d139ea6-46b2-4bbe-a46d-e641a952d200 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:27.457974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c6bdb-8d2c-4449-869f-2a7db47aa6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:27.458775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c6bdb-8d2c-4449-869f-2a7db47aa6e4 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:27.459060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c6bdb-8d2c-4449-869f-2a7db47aa6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:27.459335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c6bdb-8d2c-4449-869f-2a7db47aa6e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:27.470147 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c6bdb-8d2c-4449-869f-2a7db47aa6e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:27.472157 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c6bdb-8d2c-4449-869f-2a7db47aa6e4 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-ac3c6bdb-8d2c-4449-869f-2a7db47aa6e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:27.489800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:27.490891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:27.492057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:27.492057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:27.492057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:27.492057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:27.492057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 459/933] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:27.498136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0bc65-1590-4e1f-b3c6-28abcbd05a63 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:27.519066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:27.605058 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0bc65-1590-4e1f-b3c6-28abcbd05a63 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:27.641156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b00dc-1810-4daf-886b-81503bd04dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:27.642162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b00dc-1810-4daf-886b-81503bd04dd8 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:27.643187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b00dc-1810-4daf-886b-81503bd04dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:27.643646 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b00dc-1810-4daf-886b-81503bd04dd8 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:27.644808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b00dc-1810-4daf-886b-81503bd04dd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:27.645343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b00dc-1810-4daf-886b-81503bd04dd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:27.646960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b00dc-1810-4daf-886b-81503bd04dd8 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:27.664442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 475/934] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:27.666613 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c6bdb-8d2c-4449-869f-2a7db47aa6e4 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:27.774747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c6bdb-8d2c-4449-869f-2a7db47aa6e4 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:27.791608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:27.809701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e78c7-a741-402d-9a9f-7edca841e13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:27.810725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e78c7-a741-402d-9a9f-7edca841e13d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:27.811011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e78c7-a741-402d-9a9f-7edca841e13d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:27.811285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e78c7-a741-402d-9a9f-7edca841e13d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:27.818505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e78c7-a741-402d-9a9f-7edca841e13d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:27.819980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e78c7-a741-402d-9a9f-7edca841e13d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-dc7e78c7-a741-402d-9a9f-7edca841e13d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:27.834717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:27.835593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:27.836647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:27.836647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:27.836647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:27.836647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:27.836647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:27.836647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:27.836647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:27.836647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:27.836647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 460/935] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:27.843397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b00dc-1810-4daf-886b-81503bd04dd8 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:27.925518 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:27.947836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b00dc-1810-4daf-886b-81503bd04dd8 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:27.982222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c5b66-b99d-4603-bcef-1de2e7cef0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:27.982962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c5b66-b99d-4603-bcef-1de2e7cef0f4 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:27.983252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c5b66-b99d-4603-bcef-1de2e7cef0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:27.983527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c5b66-b99d-4603-bcef-1de2e7cef0f4 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:27.984312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c5b66-b99d-4603-bcef-1de2e7cef0f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:27.984643 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c5b66-b99d-4603-bcef-1de2e7cef0f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:27.986597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c5b66-b99d-4603-bcef-1de2e7cef0f4 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:27.999312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 476/936] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:28.005853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e78c7-a741-402d-9a9f-7edca841e13d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:28.091426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889b099-8d36-4864-90c3-9e3c0954b0e9 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:28.091634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889b099-8d36-4864-90c3-9e3c0954b0e9 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:28.091901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889b099-8d36-4864-90c3-9e3c0954b0e9 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:28.093315 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 477/937] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:28.098445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed14c2-4f7c-4931-82bd-f954ffb422ef None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:28.098811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed14c2-4f7c-4931-82bd-f954ffb422ef None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:28.099164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed14c2-4f7c-4931-82bd-f954ffb422ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:28.116643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e78c7-a741-402d-9a9f-7edca841e13d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:28.118096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-04ed14c2-4f7c-4931-82bd-f954ffb422ef None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:28.119364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-04ed14c2-4f7c-4931-82bd-f954ffb422ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:28.154378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46df2b-e037-439b-9d5d-b567bbc37aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:28.155688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46df2b-e037-439b-9d5d-b567bbc37aff None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:28.156154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46df2b-e037-439b-9d5d-b567bbc37aff None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:28.156691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46df2b-e037-439b-9d5d-b567bbc37aff None admin] PATH_INFO: `/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/c151c53202f24faeb3accf802716652f/roles/1ac8ff72ed404116b52f57f20668457d` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:28.159145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46df2b-e037-439b-9d5d-b567bbc37aff None admin] RBAC: Authorizing `identity:create_grant(project_id=2ee27c4da77c47428e02fd65e890d256, user_id=c151c53202f24faeb3accf802716652f, role_id=1ac8ff72ed404116b52f57f20668457d)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:28.160852 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46df2b-e037-439b-9d5d-b567bbc37aff None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:28.185029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 461/938] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:33:28 2022] PUT /identity/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/c151c53202f24faeb3accf802716652f/roles/1ac8ff72ed404116b52f57f20668457d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:33:28.197825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 478/939] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:28.220506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46df2b-e037-439b-9d5d-b567bbc37aff None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:28.259851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:28.344902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46df2b-e037-439b-9d5d-b567bbc37aff None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:28.380830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f505140-9342-4b1a-a489-8bb7e9766942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:28.381737 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f505140-9342-4b1a-a489-8bb7e9766942 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:28.382032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f505140-9342-4b1a-a489-8bb7e9766942 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:28.382358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f505140-9342-4b1a-a489-8bb7e9766942 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:28.383027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f505140-9342-4b1a-a489-8bb7e9766942 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:28.384449 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f505140-9342-4b1a-a489-8bb7e9766942 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-9f505140-9342-4b1a-a489-8bb7e9766942 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:28.397844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:28.399127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:28.400029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 462/940] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Jul 25 08:33:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:28.405316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed14c2-4f7c-4931-82bd-f954ffb422ef None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:28.476270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:28.514840 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed14c2-4f7c-4931-82bd-f954ffb422ef None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:28.553088 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e02d13-f9b0-4238-93d7-80122ca6620e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:28.553770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e02d13-f9b0-4238-93d7-80122ca6620e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:28.554056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e02d13-f9b0-4238-93d7-80122ca6620e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:28.554336 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e02d13-f9b0-4238-93d7-80122ca6620e None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:28.555061 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e02d13-f9b0-4238-93d7-80122ca6620e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:28.555368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e02d13-f9b0-4238-93d7-80122ca6620e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:28.556778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e02d13-f9b0-4238-93d7-80122ca6620e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:28.562366 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 479/941] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Jul 25 08:33:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:28.571563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f505140-9342-4b1a-a489-8bb7e9766942 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:28.584757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f3f09-cf25-4f63-be5b-36e8cd50ef20 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:28.585258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f3f09-cf25-4f63-be5b-36e8cd50ef20 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:28.585636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f3f09-cf25-4f63-be5b-36e8cd50ef20 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:28.587539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 480/942] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:28.593124 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ba888-b3b3-48b1-a484-68e26234f11e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:28.593428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ba888-b3b3-48b1-a484-68e26234f11e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:28.593712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ba888-b3b3-48b1-a484-68e26234f11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:28.618191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-239ba888-b3b3-48b1-a484-68e26234f11e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:28.619340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-239ba888-b3b3-48b1-a484-68e26234f11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:28.679782 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f505140-9342-4b1a-a489-8bb7e9766942 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:28.687214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:28.715581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91743610-603b-44c8-bd33-582c9f4cb196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:28.716399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91743610-603b-44c8-bd33-582c9f4cb196 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:28.716829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91743610-603b-44c8-bd33-582c9f4cb196 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:28.716941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 481/943] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:28.717302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91743610-603b-44c8-bd33-582c9f4cb196 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:28.718125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91743610-603b-44c8-bd33-582c9f4cb196 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:28.718586 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91743610-603b-44c8-bd33-582c9f4cb196 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:28.720154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91743610-603b-44c8-bd33-582c9f4cb196 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:28.726227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 463/944] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Jul 25 08:33:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:28.732079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91743610-603b-44c8-bd33-582c9f4cb196 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:28.732218 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ba888-b3b3-48b1-a484-68e26234f11e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:28.795471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:28.835442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ba888-b3b3-48b1-a484-68e26234f11e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:28.839648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91743610-603b-44c8-bd33-582c9f4cb196 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:28.871325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d511d2-7d4f-4a8c-a380-fffb081d2def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:28.871915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d511d2-7d4f-4a8c-a380-fffb081d2def None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:28.872198 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d511d2-7d4f-4a8c-a380-fffb081d2def None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:28.872474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d511d2-7d4f-4a8c-a380-fffb081d2def None admin] PATH_INFO: `/v3/endpoints` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:28.873302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d511d2-7d4f-4a8c-a380-fffb081d2def None admin] RBAC: Adding query filter params (interface=public, service_id=52188dacf7cc4ee49197913fa4a93f50, region_id=RegionOne) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:28.873617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d511d2-7d4f-4a8c-a380-fffb081d2def None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:28.874998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d511d2-7d4f-4a8c-a380-fffb081d2def None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:28.877536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d1096-f246-4c82-9429-d23e0823da40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:28.878376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d1096-f246-4c82-9429-d23e0823da40 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:28.878815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d1096-f246-4c82-9429-d23e0823da40 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:28.879266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d1096-f246-4c82-9429-d23e0823da40 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:28.880126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d1096-f246-4c82-9429-d23e0823da40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:28.881809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d1096-f246-4c82-9429-d23e0823da40 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:28.884716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: WARNING keystone.server.flask.common [None req-ee9d1096-f246-4c82-9429-d23e0823da40 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 08:33:28.884850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 482/945] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Jul 25 08:33:28 2022] GET /identity/v3/endpoints?service_id=52188dacf7cc4ee49197913fa4a93f50&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:28.886074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d511d2-7d4f-4a8c-a380-fffb081d2def None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:28.906579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.sql.core [None req-ee9d1096-f246-4c82-9429-d23e0823da40 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 08:33:28.906579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 08:33:28.906579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [parameters: {'id': '298f2fbe05e24c3e9d0f3849bbc1a317', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 08:33:28.906579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86751) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-ee9d1096-f246-4c82-9429-d23e0823da40 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 298f2fbe05e24c3e9d0f3849bbc1a317.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 298f2fbe05e24c3e9d0f3849bbc1a317. Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application conn.query(q) Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 08:33:28.918558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application result.read() Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 25 08:33:28.919512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application next(self.gen) Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application yield resource Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application next(self.gen) Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise self.value Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application session.commit() Jul 25 08:33:28.920506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self.session.flush() Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application compat.raise_( Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 08:33:28.921519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 08:33:28.923275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application conn.query(q) Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application result.read() Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 08:33:28.924138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [parameters: {'id': '298f2fbe05e24c3e9d0f3849bbc1a317', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:28.926080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:28.928810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:28.931371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:28.931371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 25 08:33:28.931371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 25 08:33:28.931371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:28.931371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:28.931371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 25 08:33:28.931371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 25 08:33:28.931371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 298f2fbe05e24c3e9d0f3849bbc1a317. Jul 25 08:33:28.931371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:28.931371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 464/946] 10.222.0.52 () {60 vars in 1140 bytes} [Mon Jul 25 08:33:28 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:28.931371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d1096-f246-4c82-9429-d23e0823da40 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:28.960994 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:29.023999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d511d2-7d4f-4a8c-a380-fffb081d2def None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:29.056605 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:29.067032 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bc28a-4f0d-4965-a8d2-45532cd82478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:29.067624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d1096-f246-4c82-9429-d23e0823da40 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:29.068271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bc28a-4f0d-4965-a8d2-45532cd82478 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:29.068832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bc28a-4f0d-4965-a8d2-45532cd82478 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:29.069334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bc28a-4f0d-4965-a8d2-45532cd82478 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:29.070277 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bc28a-4f0d-4965-a8d2-45532cd82478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:29.071924 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bc28a-4f0d-4965-a8d2-45532cd82478 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:29.078767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 483/947] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Jul 25 08:33:28 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:33:29.110557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515cd3a-03f2-4ccc-b23c-2865dde7412f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:29.111827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515cd3a-03f2-4ccc-b23c-2865dde7412f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:29.112326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515cd3a-03f2-4ccc-b23c-2865dde7412f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:29.112873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515cd3a-03f2-4ccc-b23c-2865dde7412f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:29.121273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d515cd3a-03f2-4ccc-b23c-2865dde7412f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:29.123628 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d515cd3a-03f2-4ccc-b23c-2865dde7412f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-d515cd3a-03f2-4ccc-b23c-2865dde7412f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:29.140026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:29.141042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:29.141928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:29.141928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:29.141928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:29.141928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:29.141928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:29.141928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:29.141928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:29.141928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:29.145548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 465/948] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:29.151286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bc28a-4f0d-4965-a8d2-45532cd82478 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:29.269799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bc28a-4f0d-4965-a8d2-45532cd82478 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:29.306139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b588db-d7af-4a37-bf60-aae18355d926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:29.306405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b588db-d7af-4a37-bf60-aae18355d926 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:29.306625 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b588db-d7af-4a37-bf60-aae18355d926 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:29.306905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b588db-d7af-4a37-bf60-aae18355d926 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:29.307675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b588db-d7af-4a37-bf60-aae18355d926 None admin] RBAC: Adding query filter params (name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:29.307982 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b588db-d7af-4a37-bf60-aae18355d926 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:29.309460 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b588db-d7af-4a37-bf60-aae18355d926 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:29.336754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 484/949] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:29.445954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051547a-7971-4fbf-9c05-8928108c7071 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:29.446322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051547a-7971-4fbf-9c05-8928108c7071 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:29.446508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051547a-7971-4fbf-9c05-8928108c7071 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:29.451016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 466/950] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:29.456993 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16d8b1-b344-44f0-baa9-89b1abceb706 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:29.457285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16d8b1-b344-44f0-baa9-89b1abceb706 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:29.457569 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16d8b1-b344-44f0-baa9-89b1abceb706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:29.477593 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-6f16d8b1-b344-44f0-baa9-89b1abceb706 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:29.478809 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6f16d8b1-b344-44f0-baa9-89b1abceb706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:29.528820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:29.551270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 485/951] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:29.574247 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051547a-7971-4fbf-9c05-8928108c7071 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:29.661486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:29.687523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051547a-7971-4fbf-9c05-8928108c7071 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:29.724608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fc690-7987-4c28-b524-bbd61bf25c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:29.725645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fc690-7987-4c28-b524-bbd61bf25c48 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:29.726078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fc690-7987-4c28-b524-bbd61bf25c48 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:29.726524 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fc690-7987-4c28-b524-bbd61bf25c48 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:29.727332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fc690-7987-4c28-b524-bbd61bf25c48 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:29.728871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fc690-7987-4c28-b524-bbd61bf25c48 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-7f4fc690-7987-4c28-b524-bbd61bf25c48 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:29.741741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:29.743006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 08:33:29.744126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:29.746695 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 467/952] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:29.754587 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16d8b1-b344-44f0-baa9-89b1abceb706 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:29.858171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16d8b1-b344-44f0-baa9-89b1abceb706 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:29.893743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fff9b-7c85-4b5c-a177-fe71f5121968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:29.894523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fff9b-7c85-4b5c-a177-fe71f5121968 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:29.894818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fff9b-7c85-4b5c-a177-fe71f5121968 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:29.895095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fff9b-7c85-4b5c-a177-fe71f5121968 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:29.895812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fff9b-7c85-4b5c-a177-fe71f5121968 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:29.896122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fff9b-7c85-4b5c-a177-fe71f5121968 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:29.897534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fff9b-7c85-4b5c-a177-fe71f5121968 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:29.903057 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 486/953] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:29.909403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fc690-7987-4c28-b524-bbd61bf25c48 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:30.022322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fc690-7987-4c28-b524-bbd61bf25c48 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:30.058139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1f449-bf97-4865-bf9d-be9bc3c6fb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:30.059076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1f449-bf97-4865-bf9d-be9bc3c6fb0f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:30.059523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1f449-bf97-4865-bf9d-be9bc3c6fb0f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:30.059963 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1f449-bf97-4865-bf9d-be9bc3c6fb0f None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:30.060870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1f449-bf97-4865-bf9d-be9bc3c6fb0f None admin] RBAC: Adding query filter params (type=compute) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:30.061340 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1f449-bf97-4865-bf9d-be9bc3c6fb0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:30.062852 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1f449-bf97-4865-bf9d-be9bc3c6fb0f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:30.068692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 468/954] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:30.238251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da8d63-f880-4536-b41d-c798567f7b58 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:30.238538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da8d63-f880-4536-b41d-c798567f7b58 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:30.238821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da8d63-f880-4536-b41d-c798567f7b58 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:30.240445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 487/955] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:30.245584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423ca3-7094-4eb4-87e3-ecb2b8aa4a48 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:30.245871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423ca3-7094-4eb4-87e3-ecb2b8aa4a48 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:30.246150 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423ca3-7094-4eb4-87e3-ecb2b8aa4a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:30.265560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-1d423ca3-7094-4eb4-87e3-ecb2b8aa4a48 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:30.267074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1d423ca3-7094-4eb4-87e3-ecb2b8aa4a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:30.269346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:30.339754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 469/956] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:30.362405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da8d63-f880-4536-b41d-c798567f7b58 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:30.412069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:30.467772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da8d63-f880-4536-b41d-c798567f7b58 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:30.503837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a43c9f-2d98-4de6-a912-7cd115d84ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:30.504690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a43c9f-2d98-4de6-a912-7cd115d84ee9 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:30.504978 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a43c9f-2d98-4de6-a912-7cd115d84ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:30.505260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a43c9f-2d98-4de6-a912-7cd115d84ee9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:30.517856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a43c9f-2d98-4de6-a912-7cd115d84ee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-80a43c9f-2d98-4de6-a912-7cd115d84ee9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:30.526436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:30.527713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:30.528610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 488/957] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Jul 25 08:33:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:30.537833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d423ca3-7094-4eb4-87e3-ecb2b8aa4a48 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:30.640273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d423ca3-7094-4eb4-87e3-ecb2b8aa4a48 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:30.680220 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f658f2-912c-42ea-8619-a1fc3206999e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:30.680930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f658f2-912c-42ea-8619-a1fc3206999e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:30.681218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f658f2-912c-42ea-8619-a1fc3206999e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:30.681500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f658f2-912c-42ea-8619-a1fc3206999e None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:30.682254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f658f2-912c-42ea-8619-a1fc3206999e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:30.682562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f658f2-912c-42ea-8619-a1fc3206999e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:30.683936 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f658f2-912c-42ea-8619-a1fc3206999e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:30.692459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 470/958] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Jul 25 08:33:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:30.693442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:30.697618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a43c9f-2d98-4de6-a912-7cd115d84ee9 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:30.803511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a43c9f-2d98-4de6-a912-7cd115d84ee9 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:30.806276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:30.840953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767e9e4-4512-4330-a19a-ae624be4dd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:30.842098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767e9e4-4512-4330-a19a-ae624be4dd98 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:30.842575 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767e9e4-4512-4330-a19a-ae624be4dd98 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:30.843043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767e9e4-4512-4330-a19a-ae624be4dd98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:30.853971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767e9e4-4512-4330-a19a-ae624be4dd98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:30.855681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767e9e4-4512-4330-a19a-ae624be4dd98 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-7767e9e4-4512-4330-a19a-ae624be4dd98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:30.873762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:30.874702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:30.875604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:30.875604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:30.875604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:30.875604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:30.878192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 489/959] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:30.885813 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f658f2-912c-42ea-8619-a1fc3206999e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:30.995344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f658f2-912c-42ea-8619-a1fc3206999e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:31.030134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6030f08-1c56-4f38-98ae-1bce9b682a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:31.031156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6030f08-1c56-4f38-98ae-1bce9b682a7d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:31.031156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6030f08-1c56-4f38-98ae-1bce9b682a7d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:31.031428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6030f08-1c56-4f38-98ae-1bce9b682a7d None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:31.032079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6030f08-1c56-4f38-98ae-1bce9b682a7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:31.032388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6030f08-1c56-4f38-98ae-1bce9b682a7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:31.033824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6030f08-1c56-4f38-98ae-1bce9b682a7d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:31.047507 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 471/960] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:31.053451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767e9e4-4512-4330-a19a-ae624be4dd98 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:31.164700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767e9e4-4512-4330-a19a-ae624be4dd98 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:31.201082 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d346cd1-6501-4cfb-84d0-afa2ab0499a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:31.202264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d346cd1-6501-4cfb-84d0-afa2ab0499a3 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:31.202750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d346cd1-6501-4cfb-84d0-afa2ab0499a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:31.203242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d346cd1-6501-4cfb-84d0-afa2ab0499a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:31.211565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d346cd1-6501-4cfb-84d0-afa2ab0499a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:31.213309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d346cd1-6501-4cfb-84d0-afa2ab0499a3 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-1d346cd1-6501-4cfb-84d0-afa2ab0499a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:31.229152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:31.230188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:31.231128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:31.231128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:31.231128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:31.231128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:31.231128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:31.231128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:31.231128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:31.231128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:31.231128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 08:33:31.231128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:31.233957 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 490/961] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Jul 25 08:33:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:31.240489 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6030f08-1c56-4f38-98ae-1bce9b682a7d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:31.306825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:31.316903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f91aa3-5618-450b-ab6c-605e57cf083f None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:31.317502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f91aa3-5618-450b-ab6c-605e57cf083f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:31.318012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f91aa3-5618-450b-ab6c-605e57cf083f None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:31.319887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 491/962] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:31.325693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2946b-8064-4280-b5b9-a6e89e5fdef2 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:31.325980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2946b-8064-4280-b5b9-a6e89e5fdef2 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:31.326255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2946b-8064-4280-b5b9-a6e89e5fdef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:31.344777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7da2946b-8064-4280-b5b9-a6e89e5fdef2 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:31.346488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7da2946b-8064-4280-b5b9-a6e89e5fdef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:31.349940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6030f08-1c56-4f38-98ae-1bce9b682a7d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:31.357947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:31.384993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b344c40-8f6d-4261-ab8e-c48e97ff6a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:31.385677 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344c40-8f6d-4261-ab8e-c48e97ff6a51 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:31.385964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344c40-8f6d-4261-ab8e-c48e97ff6a51 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:31.386240 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344c40-8f6d-4261-ab8e-c48e97ff6a51 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:31.387081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b344c40-8f6d-4261-ab8e-c48e97ff6a51 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:31.387391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b344c40-8f6d-4261-ab8e-c48e97ff6a51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:31.388861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b344c40-8f6d-4261-ab8e-c48e97ff6a51 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:31.411670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 472/963] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Jul 25 08:33:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:31.413065 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b344c40-8f6d-4261-ab8e-c48e97ff6a51 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:31.432756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 492/964] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:31.432756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01021e-a193-4f85-aa0b-d696f63e4b95 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:31.432756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01021e-a193-4f85-aa0b-d696f63e4b95 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:31.432756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01021e-a193-4f85-aa0b-d696f63e4b95 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:31.433312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 493/965] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:31.438790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed01e81-af38-4f6d-bc83-5998626c50f4 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:31.439117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed01e81-af38-4f6d-bc83-5998626c50f4 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:31.439552 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed01e81-af38-4f6d-bc83-5998626c50f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:31.460152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-0ed01e81-af38-4f6d-bc83-5998626c50f4 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:31.461468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0ed01e81-af38-4f6d-bc83-5998626c50f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:31.516649 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b344c40-8f6d-4261-ab8e-c48e97ff6a51 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:31.535190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:31.544905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 494/966] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:31.546920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed01e81-af38-4f6d-bc83-5998626c50f4 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:31.557979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d82768-9af3-41fa-8313-0112037d5d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:31.558577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d82768-9af3-41fa-8313-0112037d5d25 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:31.558917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d82768-9af3-41fa-8313-0112037d5d25 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:31.559268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d82768-9af3-41fa-8313-0112037d5d25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:31.632560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d82768-9af3-41fa-8313-0112037d5d25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:31.634036 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d82768-9af3-41fa-8313-0112037d5d25 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:31.636748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:31.661110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed01e81-af38-4f6d-bc83-5998626c50f4 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:31.697703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9e965-469c-4a0f-a87e-a2b00b6f072d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:31.698524 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9e965-469c-4a0f-a87e-a2b00b6f072d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:31.698810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9e965-469c-4a0f-a87e-a2b00b6f072d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:31.699089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9e965-469c-4a0f-a87e-a2b00b6f072d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:31.699748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9e965-469c-4a0f-a87e-a2b00b6f072d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:31.701161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9e965-469c-4a0f-a87e-a2b00b6f072d None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-b8e9e965-469c-4a0f-a87e-a2b00b6f072d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:31.713657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:31.714735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:31.721364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 495/967] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Jul 25 08:33:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:31.722372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9e965-469c-4a0f-a87e-a2b00b6f072d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:31.751135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 473/968] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:31 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:33:31.752093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d82768-9af3-41fa-8313-0112037d5d25 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:31.770872 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:31.834483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9e965-469c-4a0f-a87e-a2b00b6f072d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:31.862010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d82768-9af3-41fa-8313-0112037d5d25 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:31.868717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075e838-cf09-4ba6-8de3-9b9540f31acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:31.869391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075e838-cf09-4ba6-8de3-9b9540f31acd None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:31.869675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075e838-cf09-4ba6-8de3-9b9540f31acd None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:31.869952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075e838-cf09-4ba6-8de3-9b9540f31acd None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:31.870654 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075e838-cf09-4ba6-8de3-9b9540f31acd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:31.872078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075e838-cf09-4ba6-8de3-9b9540f31acd None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:31.874629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: WARNING keystone.server.flask.common [None req-2075e838-cf09-4ba6-8de3-9b9540f31acd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 08:33:31.886502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.sql.core [None req-2075e838-cf09-4ba6-8de3-9b9540f31acd None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 08:33:31.886502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 08:33:31.886502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [parameters: {'user_id': 'ac30911f92ed4cfd92f70ee5bd738c96', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 08:33:31.886502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86752) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 08:33:31.899937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adabf64-8679-4d1c-8e40-9ea5891ee2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:31.900944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adabf64-8679-4d1c-8e40-9ea5891ee2e9 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:31.901682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adabf64-8679-4d1c-8e40-9ea5891ee2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:31.902157 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adabf64-8679-4d1c-8e40-9ea5891ee2e9 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:31.903071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adabf64-8679-4d1c-8e40-9ea5891ee2e9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-2075e838-cf09-4ba6-8de3-9b9540f31acd None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application conn.query(q) Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application result.read() Jul 25 08:33:31.903367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 08:33:31.904245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application next(self.gen) Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application yield resource Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application next(self.gen) Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise self.value Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application session.commit() Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 08:33:31.905259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self.session.flush() Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application compat.raise_( Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 08:33:31.906174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 08:33:31.907099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application conn.query(q) Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application result.read() Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 08:33:31.908537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ac30911f92ed4cfd92f70ee5bd738c96', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:31.911369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 25 08:33:31.913367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 25 08:33:31.915475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 25 08:33:31.915475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 25 08:33:31.915475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 08:33:31.915475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 08:33:31.915475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 08:33:31.915475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:31.915475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 496/969] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Jul 25 08:33:31 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:31.915475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075e838-cf09-4ba6-8de3-9b9540f31acd None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:31.917442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adabf64-8679-4d1c-8e40-9ea5891ee2e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:31.917442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adabf64-8679-4d1c-8e40-9ea5891ee2e9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:31.917738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:31.921098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 474/970] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Jul 25 08:33:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:31.921994 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adabf64-8679-4d1c-8e40-9ea5891ee2e9 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:32.011513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:32.021543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075e838-cf09-4ba6-8de3-9b9540f31acd None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:32.034309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adabf64-8679-4d1c-8e40-9ea5891ee2e9 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:32.058548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a6b65-f03b-4c00-beb1-3de2bbf05101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:32.059346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a6b65-f03b-4c00-beb1-3de2bbf05101 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:32.059627 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a6b65-f03b-4c00-beb1-3de2bbf05101 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:32.059901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a6b65-f03b-4c00-beb1-3de2bbf05101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:32.070982 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a6b65-f03b-4c00-beb1-3de2bbf05101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:32.071632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594dd57-3ccb-4473-b0e0-d6d250a9b1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:32.072656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a6b65-f03b-4c00-beb1-3de2bbf05101 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:32.072849 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594dd57-3ccb-4473-b0e0-d6d250a9b1e4 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:32.073471 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594dd57-3ccb-4473-b0e0-d6d250a9b1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:32.073938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594dd57-3ccb-4473-b0e0-d6d250a9b1e4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:32.083225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594dd57-3ccb-4473-b0e0-d6d250a9b1e4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:32.085039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594dd57-3ccb-4473-b0e0-d6d250a9b1e4 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-161a6b65-f03b-4c00-beb1-3de2bbf05101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:32.091630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:32.092504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:32.093467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:32.093467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:32.093467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:32.093467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:32.097557 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 497/971] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:32.098170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a6b65-f03b-4c00-beb1-3de2bbf05101 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-3594dd57-3ccb-4473-b0e0-d6d250a9b1e4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:32.101416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:32.102324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:32.103690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:32.103690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:32.103690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:32.103690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:32.103690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:32.103690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:32.103690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:32.103690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:32.103690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 08:33:32.103690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:32.108293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 475/972] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:33:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:32.109260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594dd57-3ccb-4473-b0e0-d6d250a9b1e4 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:32.158015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:32.203955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a6b65-f03b-4c00-beb1-3de2bbf05101 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:32.215428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594dd57-3ccb-4473-b0e0-d6d250a9b1e4 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:32.237209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bdee6-8dfb-4f08-87e7-679f71f1d425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:32.237869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bdee6-8dfb-4f08-87e7-679f71f1d425 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:32.238154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bdee6-8dfb-4f08-87e7-679f71f1d425 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:32.238433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bdee6-8dfb-4f08-87e7-679f71f1d425 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:32.239140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7bdee6-8dfb-4f08-87e7-679f71f1d425 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:32.239442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7bdee6-8dfb-4f08-87e7-679f71f1d425 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:32.240839 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7bdee6-8dfb-4f08-87e7-679f71f1d425 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:32.247795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237b831-077c-491a-b0e3-c1d6304ebcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:32.248453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 498/973] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Jul 25 08:33:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:32.248696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237b831-077c-491a-b0e3-c1d6304ebcc0 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:32.248805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237b831-077c-491a-b0e3-c1d6304ebcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:32.249029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237b831-077c-491a-b0e3-c1d6304ebcc0 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:32.249335 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bdee6-8dfb-4f08-87e7-679f71f1d425 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:32.249812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237b831-077c-491a-b0e3-c1d6304ebcc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:32.250124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237b831-077c-491a-b0e3-c1d6304ebcc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:32.251968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237b831-077c-491a-b0e3-c1d6304ebcc0 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:32.264076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 476/974] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:32.265411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a7bc6-aad8-4c70-8217-9da56b510c04 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:32.265703 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a7bc6-aad8-4c70-8217-9da56b510c04 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:32.265980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a7bc6-aad8-4c70-8217-9da56b510c04 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:32.267787 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 477/975] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:32.268458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a7bc6-aad8-4c70-8217-9da56b510c04 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:32.271200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:32.358642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bdee6-8dfb-4f08-87e7-679f71f1d425 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:32.373147 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a7bc6-aad8-4c70-8217-9da56b510c04 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:32.374829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:32.393141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b976d-0c9a-4b1a-aec9-fc647c8d2343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:32.393812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b976d-0c9a-4b1a-aec9-fc647c8d2343 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:32.394096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b976d-0c9a-4b1a-aec9-fc647c8d2343 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:32.394372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b976d-0c9a-4b1a-aec9-fc647c8d2343 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:32.395171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b976d-0c9a-4b1a-aec9-fc647c8d2343 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:32.395475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b976d-0c9a-4b1a-aec9-fc647c8d2343 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:32.396951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b976d-0c9a-4b1a-aec9-fc647c8d2343 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:32.408294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd7acd-bf75-40eb-89b8-bf824c4052cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:32.408977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd7acd-bf75-40eb-89b8-bf824c4052cc None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:32.409258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd7acd-bf75-40eb-89b8-bf824c4052cc None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:32.409530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd7acd-bf75-40eb-89b8-bf824c4052cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:32.410268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd7acd-bf75-40eb-89b8-bf824c4052cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:32.411600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd7acd-bf75-40eb-89b8-bf824c4052cc None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:32.431609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 499/976] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Jul 25 08:33:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:32.432096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 478/977] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:33:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:32.432436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b976d-0c9a-4b1a-aec9-fc647c8d2343 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:32.433772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cdf6f-329f-4298-bfbe-aaf42d34f018 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:32.434068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cdf6f-329f-4298-bfbe-aaf42d34f018 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:32.434345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cdf6f-329f-4298-bfbe-aaf42d34f018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:32.460207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-948cdf6f-329f-4298-bfbe-aaf42d34f018 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:32.461794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-948cdf6f-329f-4298-bfbe-aaf42d34f018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:32.533089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:32.564607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 479/978] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:32.581590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b976d-0c9a-4b1a-aec9-fc647c8d2343 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:32.591690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cdf6f-329f-4298-bfbe-aaf42d34f018 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:32.633114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7b29d-d214-4b93-83ba-7ab236f80668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:32.634514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7b29d-d214-4b93-83ba-7ab236f80668 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:32.635423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7b29d-d214-4b93-83ba-7ab236f80668 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:32.636266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7b29d-d214-4b93-83ba-7ab236f80668 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:32.649246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7b29d-d214-4b93-83ba-7ab236f80668 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:32.651988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7b29d-d214-4b93-83ba-7ab236f80668 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-6ea7b29d-d214-4b93-83ba-7ab236f80668 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:32.676149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:32.677221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:32.678118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:32.678118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:32.678118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:32.678118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:32.678118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:32.678118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:32.678118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:32.678118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:32.678565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 500/979] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:32.686521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7b29d-d214-4b93-83ba-7ab236f80668 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:32.730351 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:32.736391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cdf6f-329f-4298-bfbe-aaf42d34f018 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:32.773394 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b980f-d7be-42c0-967a-5d930c2d3167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:32.774063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b980f-d7be-42c0-967a-5d930c2d3167 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:32.774340 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b980f-d7be-42c0-967a-5d930c2d3167 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:32.774621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b980f-d7be-42c0-967a-5d930c2d3167 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:32.775471 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b980f-d7be-42c0-967a-5d930c2d3167 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:32.777261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b980f-d7be-42c0-967a-5d930c2d3167 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:32.796744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 480/980] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:33:32 2022] POST /identity/v3/services => generated 242 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:33:32.800883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7b29d-d214-4b93-83ba-7ab236f80668 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:32.846289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239345e9-6263-478d-94c7-c50aacec11be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:32.846974 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239345e9-6263-478d-94c7-c50aacec11be None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:32.847257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239345e9-6263-478d-94c7-c50aacec11be None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:32.847530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239345e9-6263-478d-94c7-c50aacec11be None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:32.848366 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239345e9-6263-478d-94c7-c50aacec11be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:32.848718 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239345e9-6263-478d-94c7-c50aacec11be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:32.850314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239345e9-6263-478d-94c7-c50aacec11be None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:32.868902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 501/981] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:32.870483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b980f-d7be-42c0-967a-5d930c2d3167 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:32.922346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:32.991291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b980f-d7be-42c0-967a-5d930c2d3167 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:33.025996 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56771afe-43c1-4d5c-b8b4-1e2d1d91a01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:33.026913 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56771afe-43c1-4d5c-b8b4-1e2d1d91a01a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:33.027196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56771afe-43c1-4d5c-b8b4-1e2d1d91a01a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:33.027469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56771afe-43c1-4d5c-b8b4-1e2d1d91a01a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:33.028297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56771afe-43c1-4d5c-b8b4-1e2d1d91a01a None admin] RBAC: Adding query filter params (role.id=1ac8ff72ed404116b52f57f20668457d, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=c151c53202f24faeb3accf802716652f) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:33.028633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56771afe-43c1-4d5c-b8b4-1e2d1d91a01a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:33.030178 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56771afe-43c1-4d5c-b8b4-1e2d1d91a01a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:33.037687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 481/982] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:32 2022] GET /identity/v3/role_assignments?user.id=c151c53202f24faeb3accf802716652f&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=1ac8ff72ed404116b52f57f20668457d => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:34.530031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774db7a-973c-4fad-b8fc-7a81a7502068 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:34.530572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774db7a-973c-4fad-b8fc-7a81a7502068 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:34.530572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774db7a-973c-4fad-b8fc-7a81a7502068 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:34.532136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 502/983] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:34.537586 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1a1be-09c6-4d64-a122-6b76ec2d555b None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:34.537962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1a1be-09c6-4d64-a122-6b76ec2d555b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:34.538154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1a1be-09c6-4d64-a122-6b76ec2d555b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:34.557079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-bcc1a1be-09c6-4d64-a122-6b76ec2d555b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:34.558197 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-bcc1a1be-09c6-4d64-a122-6b76ec2d555b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:34.628401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 482/984] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:34.650580 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774db7a-973c-4fad-b8fc-7a81a7502068 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:34.745244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:34.756364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774db7a-973c-4fad-b8fc-7a81a7502068 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:34.792004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef73b6-8434-4454-9d21-ad9a96f07e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:34.792842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef73b6-8434-4454-9d21-ad9a96f07e72 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:34.793132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef73b6-8434-4454-9d21-ad9a96f07e72 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:34.793412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef73b6-8434-4454-9d21-ad9a96f07e72 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:34.794066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef73b6-8434-4454-9d21-ad9a96f07e72 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:34.795465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef73b6-8434-4454-9d21-ad9a96f07e72 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-54ef73b6-8434-4454-9d21-ad9a96f07e72 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:34.807534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:34.808523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:34.809427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:34.810208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614e1dc-ed14-4268-b303-af05be25ae3c None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:34.810208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614e1dc-ed14-4268-b303-af05be25ae3c None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:34.810208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614e1dc-ed14-4268-b303-af05be25ae3c None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:34.812814 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 483/985] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:34.813048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 503/986] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Jul 25 08:33:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:34.814248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614e1dc-ed14-4268-b303-af05be25ae3c None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:34.816428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895eac2-b059-4b55-9dbe-0d9a95be0136 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:34.816759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895eac2-b059-4b55-9dbe-0d9a95be0136 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:34.817038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895eac2-b059-4b55-9dbe-0d9a95be0136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:34.836824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-9895eac2-b059-4b55-9dbe-0d9a95be0136 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:34.838353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9895eac2-b059-4b55-9dbe-0d9a95be0136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:34.857522 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:34.916903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 504/987] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:34.920944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614e1dc-ed14-4268-b303-af05be25ae3c None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:34.932949 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895eac2-b059-4b55-9dbe-0d9a95be0136 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:34.958081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef463f-2113-46b1-a5f6-0f6119e2ffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:34.958751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef463f-2113-46b1-a5f6-0f6119e2ffaa None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:34.959039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef463f-2113-46b1-a5f6-0f6119e2ffaa None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:34.959320 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef463f-2113-46b1-a5f6-0f6119e2ffaa None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:34.960034 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef463f-2113-46b1-a5f6-0f6119e2ffaa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:34.960343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef463f-2113-46b1-a5f6-0f6119e2ffaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:34.961757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef463f-2113-46b1-a5f6-0f6119e2ffaa None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:34.967237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 484/988] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Jul 25 08:33:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:34.973408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef463f-2113-46b1-a5f6-0f6119e2ffaa None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:35.028521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:35.047476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895eac2-b059-4b55-9dbe-0d9a95be0136 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:35.081052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef463f-2113-46b1-a5f6-0f6119e2ffaa None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:35.082542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:35.083433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57abaa18-1e44-4c98-83f2-680f90ff9f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:35.084708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abaa18-1e44-4c98-83f2-680f90ff9f27 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.085074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abaa18-1e44-4c98-83f2-680f90ff9f27 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.085354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abaa18-1e44-4c98-83f2-680f90ff9f27 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.086041 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57abaa18-1e44-4c98-83f2-680f90ff9f27 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:35.087418 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57abaa18-1e44-4c98-83f2-680f90ff9f27 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-57abaa18-1e44-4c98-83f2-680f90ff9f27 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:35.099898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:35.101049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:35.101938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 505/989] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:35.102737 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85ae26-ef87-4214-b4f1-69fa65dede1c None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.102914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85ae26-ef87-4214-b4f1-69fa65dede1c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.103148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85ae26-ef87-4214-b4f1-69fa65dede1c None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.104418 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 506/990] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:35.107221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85ae26-ef87-4214-b4f1-69fa65dede1c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:35.118549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0033a-d948-4242-b7dc-e40ca18894e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:35.119226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0033a-d948-4242-b7dc-e40ca18894e8 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.119513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0033a-d948-4242-b7dc-e40ca18894e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.119799 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0033a-d948-4242-b7dc-e40ca18894e8 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.120511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0033a-d948-4242-b7dc-e40ca18894e8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:35.120860 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0033a-d948-4242-b7dc-e40ca18894e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:35.122227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0033a-d948-4242-b7dc-e40ca18894e8 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:35.127008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 485/991] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Jul 25 08:33:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:35.128157 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0048d-de00-4dc4-baa3-a4cae27bd5ef None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.128447 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0048d-de00-4dc4-baa3-a4cae27bd5ef None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.128769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0048d-de00-4dc4-baa3-a4cae27bd5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.145823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-18d0048d-de00-4dc4-baa3-a4cae27bd5ef None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:35.146997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-18d0048d-de00-4dc4-baa3-a4cae27bd5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:35.209398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85ae26-ef87-4214-b4f1-69fa65dede1c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:35.214958 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 486/992] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:35.215785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0048d-de00-4dc4-baa3-a4cae27bd5ef None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:35.245121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a053e-81c9-4f04-b503-b013c2f971a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:35.245850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a053e-81c9-4f04-b503-b013c2f971a7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.246139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a053e-81c9-4f04-b503-b013c2f971a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.246427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a053e-81c9-4f04-b503-b013c2f971a7 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.247166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a053e-81c9-4f04-b503-b013c2f971a7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:35.247476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a053e-81c9-4f04-b503-b013c2f971a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:35.248875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a053e-81c9-4f04-b503-b013c2f971a7 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:35.258189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 507/993] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:35.258189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a053e-81c9-4f04-b503-b013c2f971a7 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:35.269052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:35.316296 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:35.323177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0048d-de00-4dc4-baa3-a4cae27bd5ef None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:35.354384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a053e-81c9-4f04-b503-b013c2f971a7 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:35.356843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacbc24-583b-4760-927a-8ae0ea42c620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:35.357601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacbc24-583b-4760-927a-8ae0ea42c620 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.357898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacbc24-583b-4760-927a-8ae0ea42c620 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.358181 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacbc24-583b-4760-927a-8ae0ea42c620 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.358989 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacbc24-583b-4760-927a-8ae0ea42c620 None admin] RBAC: Adding query filter params (interface=admin, service_id=52188dacf7cc4ee49197913fa4a93f50, region_id=RegionOne) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:35.359309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacbc24-583b-4760-927a-8ae0ea42c620 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:35.361276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacbc24-583b-4760-927a-8ae0ea42c620 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:35.366509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 487/994] 10.222.0.52 () {56 vars in 1249 bytes} [Mon Jul 25 08:33:35 2022] GET /identity/v3/endpoints?service_id=52188dacf7cc4ee49197913fa4a93f50&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:35.367126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacbc24-583b-4760-927a-8ae0ea42c620 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:35.392361 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157669f-9bb7-48c0-8f80-dfe7da019b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:35.393189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157669f-9bb7-48c0-8f80-dfe7da019b03 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.393470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157669f-9bb7-48c0-8f80-dfe7da019b03 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.393745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157669f-9bb7-48c0-8f80-dfe7da019b03 None admin] PATH_INFO: `/v3/services/image` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.394395 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157669f-9bb7-48c0-8f80-dfe7da019b03 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:35.395772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157669f-9bb7-48c0-8f80-dfe7da019b03 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-6157669f-9bb7-48c0-8f80-dfe7da019b03 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:35.407399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:35.409062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:35.409894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 508/995] 10.222.0.52 () {56 vars in 1105 bytes} [Mon Jul 25 08:33:35 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:35.410696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157669f-9bb7-48c0-8f80-dfe7da019b03 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:35.466679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacbc24-583b-4760-927a-8ae0ea42c620 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:35.489905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:35.501514 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57e822-8884-4f19-bd18-1b5fb1a36b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:35.502179 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57e822-8884-4f19-bd18-1b5fb1a36b33 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.502473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57e822-8884-4f19-bd18-1b5fb1a36b33 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.502754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57e822-8884-4f19-bd18-1b5fb1a36b33 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.503469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57e822-8884-4f19-bd18-1b5fb1a36b33 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:35.503779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57e822-8884-4f19-bd18-1b5fb1a36b33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:35.505296 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57e822-8884-4f19-bd18-1b5fb1a36b33 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:35.514820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 488/996] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:35.514820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57e822-8884-4f19-bd18-1b5fb1a36b33 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:35.515035 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157669f-9bb7-48c0-8f80-dfe7da019b03 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:35.537771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:35.546838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fb66d-fbd7-4ade-aea4-1f3efc488d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:35.547508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fb66d-fbd7-4ade-aea4-1f3efc488d4f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.547792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fb66d-fbd7-4ade-aea4-1f3efc488d4f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.548090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fb66d-fbd7-4ade-aea4-1f3efc488d4f None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.548816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013fb66d-fbd7-4ade-aea4-1f3efc488d4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:35.550181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013fb66d-fbd7-4ade-aea4-1f3efc488d4f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:35.556383 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 509/997] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Jul 25 08:33:35 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:33:35.557188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fb66d-fbd7-4ade-aea4-1f3efc488d4f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:35.620717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57e822-8884-4f19-bd18-1b5fb1a36b33 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:35.666604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c0bf8-9064-48a2-9581-c2b9f806330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:35.667297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c0bf8-9064-48a2-9581-c2b9f806330c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.667585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c0bf8-9064-48a2-9581-c2b9f806330c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.667867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c0bf8-9064-48a2-9581-c2b9f806330c None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.668611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c0bf8-9064-48a2-9581-c2b9f806330c None admin] RBAC: Adding query filter params (name=image) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:35.668928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c0bf8-9064-48a2-9581-c2b9f806330c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:35.670351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c0bf8-9064-48a2-9581-c2b9f806330c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:35.676728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 489/998] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:35.682461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c0bf8-9064-48a2-9581-c2b9f806330c None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:35.682862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fb66d-fbd7-4ade-aea4-1f3efc488d4f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:35.723519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e9f7f-de20-4c98-a892-9620d631b851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:35.724183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e9f7f-de20-4c98-a892-9620d631b851 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.724470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e9f7f-de20-4c98-a892-9620d631b851 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.724803 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e9f7f-de20-4c98-a892-9620d631b851 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.725592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e9f7f-de20-4c98-a892-9620d631b851 None admin] RBAC: Adding query filter params (interface=public, service_id=2890091da9e444278048bf10d62c4df5, region_id=RegionOne) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:35.725901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e9f7f-de20-4c98-a892-9620d631b851 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:35.727275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e9f7f-de20-4c98-a892-9620d631b851 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:35.733567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 510/999] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Jul 25 08:33:35 2022] GET /identity/v3/endpoints?service_id=2890091da9e444278048bf10d62c4df5&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:35.739006 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e9f7f-de20-4c98-a892-9620d631b851 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:35.748862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:35.783851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:35.796844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c0bf8-9064-48a2-9581-c2b9f806330c None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:35.839914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9aa29-3efc-4213-856d-ef89cd838a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:35.840901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9aa29-3efc-4213-856d-ef89cd838a1e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.841355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9aa29-3efc-4213-856d-ef89cd838a1e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.841781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9aa29-3efc-4213-856d-ef89cd838a1e None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.842653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9aa29-3efc-4213-856d-ef89cd838a1e None admin] RBAC: Adding query filter params (type=image) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:35.843116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9aa29-3efc-4213-856d-ef89cd838a1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:35.844533 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e9f7f-de20-4c98-a892-9620d631b851 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:35.844842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9aa29-3efc-4213-856d-ef89cd838a1e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:35.849582 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 490/1000] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:35.888342 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebbb66-acb2-4973-958c-a156dbbb3695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:35.889125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebbb66-acb2-4973-958c-a156dbbb3695 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:35.889408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebbb66-acb2-4973-958c-a156dbbb3695 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:35.889683 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebbb66-acb2-4973-958c-a156dbbb3695 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:35.890372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebbb66-acb2-4973-958c-a156dbbb3695 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:35.891779 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebbb66-acb2-4973-958c-a156dbbb3695 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:35.904915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 511/1001] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Jul 25 08:33:35 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:33:37.583551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da7971-f1db-40f8-953b-26ba2f99e6b5 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:37.584047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da7971-f1db-40f8-953b-26ba2f99e6b5 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:37.584047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da7971-f1db-40f8-953b-26ba2f99e6b5 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:37.585599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 491/1002] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:37.590860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fd1ad-5a96-450a-abb8-9750b88594e9 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:37.591274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fd1ad-5a96-450a-abb8-9750b88594e9 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:37.591445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fd1ad-5a96-450a-abb8-9750b88594e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:37.609670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a02fd1ad-5a96-450a-abb8-9750b88594e9 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:37.610917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a02fd1ad-5a96-450a-abb8-9750b88594e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:37.676767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:37.689236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 512/1003] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:37.705992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da7971-f1db-40f8-953b-26ba2f99e6b5 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:37.808736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da7971-f1db-40f8-953b-26ba2f99e6b5 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:37.842607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b8f25-4631-4fb6-9ab1-12a628e41530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:37.843330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b8f25-4631-4fb6-9ab1-12a628e41530 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:37.843610 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b8f25-4631-4fb6-9ab1-12a628e41530 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:37.843955 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b8f25-4631-4fb6-9ab1-12a628e41530 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:37.844640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b8f25-4631-4fb6-9ab1-12a628e41530 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:37.846031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b8f25-4631-4fb6-9ab1-12a628e41530 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:37.854938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb862165-cb71-48c9-b7a6-7b3bc60866eb None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:37.855235 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb862165-cb71-48c9-b7a6-7b3bc60866eb None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:37.855519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb862165-cb71-48c9-b7a6-7b3bc60866eb None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:37.856942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 513/1004] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-f44b8f25-4631-4fb6-9ab1-12a628e41530 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:37.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:37.858948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:37.859792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 492/1005] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Jul 25 08:33:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:37.866255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2672-9a8c-467f-aa1d-82a0a3c30dbf None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:37.866555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2672-9a8c-467f-aa1d-82a0a3c30dbf None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:37.866661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b8f25-4631-4fb6-9ab1-12a628e41530 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:37.866841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2672-9a8c-467f-aa1d-82a0a3c30dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:37.886334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:37.887278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a6ad2672-9a8c-467f-aa1d-82a0a3c30dbf None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:37.888382 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a6ad2672-9a8c-467f-aa1d-82a0a3c30dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:37.955357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 514/1006] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:37.956767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e611b49-3807-4ae8-978f-ca966e2e4d5c None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:37.957119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e611b49-3807-4ae8-978f-ca966e2e4d5c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:37.957403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e611b49-3807-4ae8-978f-ca966e2e4d5c None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:37.958791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 515/1007] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:37.964065 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba30ba-fe72-4422-a4fc-41779f335aeb None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:37.964065 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba30ba-fe72-4422-a4fc-41779f335aeb None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:37.964334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba30ba-fe72-4422-a4fc-41779f335aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:37.973864 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b8f25-4631-4fb6-9ab1-12a628e41530 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:37.981625 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-b9ba30ba-fe72-4422-a4fc-41779f335aeb None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:37.982808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b9ba30ba-fe72-4422-a4fc-41779f335aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:38.008224 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efa03f-29ae-4d60-9c46-3f863c85d130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:38.008933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efa03f-29ae-4d60-9c46-3f863c85d130 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:38.009218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efa03f-29ae-4d60-9c46-3f863c85d130 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:38.009504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efa03f-29ae-4d60-9c46-3f863c85d130 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:38.010212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efa03f-29ae-4d60-9c46-3f863c85d130 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:38.010607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efa03f-29ae-4d60-9c46-3f863c85d130 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:38.011921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efa03f-29ae-4d60-9c46-3f863c85d130 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:38.016546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 493/1008] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Jul 25 08:33:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:38.017338 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efa03f-29ae-4d60-9c46-3f863c85d130 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:38.051700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 516/1009] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:38.052535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba30ba-fe72-4422-a4fc-41779f335aeb None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:38.111862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:38.112033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:38.131090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efa03f-29ae-4d60-9c46-3f863c85d130 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:38.154191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba30ba-fe72-4422-a4fc-41779f335aeb None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:38.164144 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73345ff6-34ae-4757-878d-f96cfcdd5761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:38.164838 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73345ff6-34ae-4757-878d-f96cfcdd5761 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:38.165189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73345ff6-34ae-4757-878d-f96cfcdd5761 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:38.165400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73345ff6-34ae-4757-878d-f96cfcdd5761 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:38.166152 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73345ff6-34ae-4757-878d-f96cfcdd5761 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:38.167682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73345ff6-34ae-4757-878d-f96cfcdd5761 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:38.183751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 494/1010] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:33:38 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:38.184349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73345ff6-34ae-4757-878d-f96cfcdd5761 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:38.204836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b2b43-c03c-477a-aa72-bea9e6b08093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:38.205977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b2b43-c03c-477a-aa72-bea9e6b08093 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:38.206267 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b2b43-c03c-477a-aa72-bea9e6b08093 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:38.206562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b2b43-c03c-477a-aa72-bea9e6b08093 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:38.207327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b2b43-c03c-477a-aa72-bea9e6b08093 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:38.207652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b2b43-c03c-477a-aa72-bea9e6b08093 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:38.210027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b2b43-c03c-477a-aa72-bea9e6b08093 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:38.223185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 517/1011] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Jul 25 08:33:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:38.224547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b2b43-c03c-477a-aa72-bea9e6b08093 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:38.311787 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73345ff6-34ae-4757-878d-f96cfcdd5761 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:38.333374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b2b43-c03c-477a-aa72-bea9e6b08093 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:38.353055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d10ac4-286c-4d92-b675-8d776296b35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:38.353964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d10ac4-286c-4d92-b675-8d776296b35c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:38.354249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d10ac4-286c-4d92-b675-8d776296b35c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:38.354525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d10ac4-286c-4d92-b675-8d776296b35c None admin] PATH_INFO: `/v3/services/compute` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:38.355176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d10ac4-286c-4d92-b675-8d776296b35c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:38.356452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:38.356566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d10ac4-286c-4d92-b675-8d776296b35c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-f4d10ac4-286c-4d92-b675-8d776296b35c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:38.371519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:38.372508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:38.373403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 495/1012] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:38.378699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d10ac4-286c-4d92-b675-8d776296b35c None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:38.380205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb87b1c-57ee-4bea-b6c1-3a378db2095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:38.380904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb87b1c-57ee-4bea-b6c1-3a378db2095c None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:38.381188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb87b1c-57ee-4bea-b6c1-3a378db2095c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:38.381466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb87b1c-57ee-4bea-b6c1-3a378db2095c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:38.382215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb87b1c-57ee-4bea-b6c1-3a378db2095c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:38.382680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:38.383577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb87b1c-57ee-4bea-b6c1-3a378db2095c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:38.403300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 518/1013] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:33:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:38.494899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d10ac4-286c-4d92-b675-8d776296b35c None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:38.539227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba768ff1-5360-48fe-b903-9b456eb85118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:38.540270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba768ff1-5360-48fe-b903-9b456eb85118 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:38.540774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba768ff1-5360-48fe-b903-9b456eb85118 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:38.541251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba768ff1-5360-48fe-b903-9b456eb85118 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:38.542201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba768ff1-5360-48fe-b903-9b456eb85118 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:38.542687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba768ff1-5360-48fe-b903-9b456eb85118 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:38.544235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba768ff1-5360-48fe-b903-9b456eb85118 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:38.551085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 496/1014] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:38.556242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb87b1c-57ee-4bea-b6c1-3a378db2095c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:38.662785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb87b1c-57ee-4bea-b6c1-3a378db2095c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:38.696082 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fb7c1-3527-4c85-a44e-f1fe651af5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:38.696781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fb7c1-3527-4c85-a44e-f1fe651af5b8 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:38.697066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fb7c1-3527-4c85-a44e-f1fe651af5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:38.697340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fb7c1-3527-4c85-a44e-f1fe651af5b8 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:38.698043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fb7c1-3527-4c85-a44e-f1fe651af5b8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:38.698346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fb7c1-3527-4c85-a44e-f1fe651af5b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:38.699698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fb7c1-3527-4c85-a44e-f1fe651af5b8 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:38.705390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 519/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Jul 25 08:33:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:38.710746 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba768ff1-5360-48fe-b903-9b456eb85118 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:38.773331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:38.811695 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba768ff1-5360-48fe-b903-9b456eb85118 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:38.845751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75652554-a52c-42dd-ad8c-36cdf57baa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:38.846416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75652554-a52c-42dd-ad8c-36cdf57baa6a None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:38.846699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75652554-a52c-42dd-ad8c-36cdf57baa6a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:38.846975 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75652554-a52c-42dd-ad8c-36cdf57baa6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:38.847723 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75652554-a52c-42dd-ad8c-36cdf57baa6a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:38.849115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75652554-a52c-42dd-ad8c-36cdf57baa6a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:38.870740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 497/1016] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:33:38 2022] POST /identity/v3/endpoints => generated 334 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:40.206579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90afae-c675-4496-87c8-06e9f719f3ab None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:40.207134 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90afae-c675-4496-87c8-06e9f719f3ab None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:40.207134 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90afae-c675-4496-87c8-06e9f719f3ab None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:40.208543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 520/1017] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:40.213599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8340e-7c24-4e2c-8afa-54bd5203f57d None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:40.214010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8340e-7c24-4e2c-8afa-54bd5203f57d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:40.214173 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8340e-7c24-4e2c-8afa-54bd5203f57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:40.232789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-39b8340e-7c24-4e2c-8afa-54bd5203f57d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:40.233977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-39b8340e-7c24-4e2c-8afa-54bd5203f57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:40.315135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 498/1018] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:40.337572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90afae-c675-4496-87c8-06e9f719f3ab None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:40.386029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:40.422744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c820c4-5734-4f24-bf43-b09e06a32ffb None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:40.423189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c820c4-5734-4f24-bf43-b09e06a32ffb None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:40.423479 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c820c4-5734-4f24-bf43-b09e06a32ffb None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:40.425031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 499/1019] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:40.431285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fbc54-7c02-4ac3-9865-a4dcb3c85008 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:40.431576 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fbc54-7c02-4ac3-9865-a4dcb3c85008 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:40.431852 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fbc54-7c02-4ac3-9865-a4dcb3c85008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:40.446143 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90afae-c675-4496-87c8-06e9f719f3ab None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:40.451278 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3f8fbc54-7c02-4ac3-9865-a4dcb3c85008 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:40.452440 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3f8fbc54-7c02-4ac3-9865-a4dcb3c85008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:40.482514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40260a4f-44ed-4740-850e-5d6fc52cf74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:40.483229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40260a4f-44ed-4740-850e-5d6fc52cf74e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:40.483754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40260a4f-44ed-4740-850e-5d6fc52cf74e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:40.483754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40260a4f-44ed-4740-850e-5d6fc52cf74e None admin] PATH_INFO: `/v3/services/image` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:40.484417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40260a4f-44ed-4740-850e-5d6fc52cf74e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:40.485852 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40260a4f-44ed-4740-850e-5d6fc52cf74e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:40.496919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-40260a4f-44ed-4740-850e-5d6fc52cf74e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:40.498255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:40.499422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:40.500273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 521/1020] 10.222.0.52 () {56 vars in 1105 bytes} [Mon Jul 25 08:33:40 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:40.506019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40260a4f-44ed-4740-850e-5d6fc52cf74e None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:40.528323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 500/1021] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:40.550584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fbc54-7c02-4ac3-9865-a4dcb3c85008 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:40.615561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40260a4f-44ed-4740-850e-5d6fc52cf74e None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:40.625411 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:40.654024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fbc54-7c02-4ac3-9865-a4dcb3c85008 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:40.654219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b173419-2ed0-42b0-86b6-723bcf6dcb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:40.654715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b173419-2ed0-42b0-86b6-723bcf6dcb9d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:40.655003 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b173419-2ed0-42b0-86b6-723bcf6dcb9d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:40.655275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b173419-2ed0-42b0-86b6-723bcf6dcb9d None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:40.656108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b173419-2ed0-42b0-86b6-723bcf6dcb9d None admin] RBAC: Adding query filter params (name=image) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:40.657041 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b173419-2ed0-42b0-86b6-723bcf6dcb9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:40.657854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b173419-2ed0-42b0-86b6-723bcf6dcb9d None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:40.663091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 522/1022] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:40.670051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b173419-2ed0-42b0-86b6-723bcf6dcb9d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:40.694806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7cf0e-4e2a-47f0-9b5f-54c67018e3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:40.695606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7cf0e-4e2a-47f0-9b5f-54c67018e3e3 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:40.695895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7cf0e-4e2a-47f0-9b5f-54c67018e3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:40.696173 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7cf0e-4e2a-47f0-9b5f-54c67018e3e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:40.697382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7cf0e-4e2a-47f0-9b5f-54c67018e3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:40.698813 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7cf0e-4e2a-47f0-9b5f-54c67018e3e3 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:40.700597 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 501/1023] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:40.706256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7cf0e-4e2a-47f0-9b5f-54c67018e3e3 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:40.752059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:40.785759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b173419-2ed0-42b0-86b6-723bcf6dcb9d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:40.808283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7cf0e-4e2a-47f0-9b5f-54c67018e3e3 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:40.821288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e382a-3ff3-4b1c-839d-32041f214014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:40.822239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e382a-3ff3-4b1c-839d-32041f214014 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:40.822528 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e382a-3ff3-4b1c-839d-32041f214014 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:40.822812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e382a-3ff3-4b1c-839d-32041f214014 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:40.823688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e382a-3ff3-4b1c-839d-32041f214014 None admin] RBAC: Adding query filter params (type=image) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:40.824375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e382a-3ff3-4b1c-839d-32041f214014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:40.825775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e382a-3ff3-4b1c-839d-32041f214014 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:40.830835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 523/1024] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:40.836405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e382a-3ff3-4b1c-839d-32041f214014 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:40.844209 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536801e2-497c-4627-8c66-a5c9a2382f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:40.845301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536801e2-497c-4627-8c66-a5c9a2382f69 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:40.845731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536801e2-497c-4627-8c66-a5c9a2382f69 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:40.846151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536801e2-497c-4627-8c66-a5c9a2382f69 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:40.847056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536801e2-497c-4627-8c66-a5c9a2382f69 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:40.848795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536801e2-497c-4627-8c66-a5c9a2382f69 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:40.872820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:40.892612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 502/1025] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:33:40 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:40.943666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e382a-3ff3-4b1c-839d-32041f214014 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:40.967016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c63af-a50f-4344-9ce8-0aafe2e19238 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:40.967357 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c63af-a50f-4344-9ce8-0aafe2e19238 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:40.967590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c63af-a50f-4344-9ce8-0aafe2e19238 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:40.969058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 503/1026] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:40.974710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e3036-028c-46ad-be23-39189c6ab394 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:40.975006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e3036-028c-46ad-be23-39189c6ab394 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:40.975293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e3036-028c-46ad-be23-39189c6ab394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:40.979639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa95a4-1aa7-4fb9-be1c-246292ce2f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:40.980317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa95a4-1aa7-4fb9-be1c-246292ce2f6a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:40.980628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa95a4-1aa7-4fb9-be1c-246292ce2f6a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:40.980910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa95a4-1aa7-4fb9-be1c-246292ce2f6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:40.981718 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa95a4-1aa7-4fb9-be1c-246292ce2f6a None admin] RBAC: Adding query filter params (interface=public, service_id=23ae98d4e5e74e649de33e2427f9c7a8, region_id=RegionOne) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:40.982034 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa95a4-1aa7-4fb9-be1c-246292ce2f6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:40.983424 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa95a4-1aa7-4fb9-be1c-246292ce2f6a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:40.993473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 524/1027] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Jul 25 08:33:40 2022] GET /identity/v3/endpoints?service_id=23ae98d4e5e74e649de33e2427f9c7a8&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:40.997750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-b65e3036-028c-46ad-be23-39189c6ab394 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:40.999328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b65e3036-028c-46ad-be23-39189c6ab394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:40.999460 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa95a4-1aa7-4fb9-be1c-246292ce2f6a None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:41.082998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 504/1028] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:41.105779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e3036-028c-46ad-be23-39189c6ab394 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:41.124520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:41.130322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa95a4-1aa7-4fb9-be1c-246292ce2f6a None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:41.135065 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:41.168462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fa585-6185-4659-8eef-207d288d3954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:41.169171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fa585-6185-4659-8eef-207d288d3954 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:41.169453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fa585-6185-4659-8eef-207d288d3954 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:41.169734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fa585-6185-4659-8eef-207d288d3954 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:41.170428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fa585-6185-4659-8eef-207d288d3954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:41.171794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fa585-6185-4659-8eef-207d288d3954 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:41.179340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 525/1029] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Jul 25 08:33:40 2022] GET /identity/v3/services => generated 1957 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:33:41.218394 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e3036-028c-46ad-be23-39189c6ab394 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:41.257732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd95ea-ca8e-4ab4-be96-4615a3966c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:41.258571 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd95ea-ca8e-4ab4-be96-4615a3966c6f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:41.258852 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd95ea-ca8e-4ab4-be96-4615a3966c6f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:41.259124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd95ea-ca8e-4ab4-be96-4615a3966c6f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:41.271032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd95ea-ca8e-4ab4-be96-4615a3966c6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-f2fd95ea-ca8e-4ab4-be96-4615a3966c6f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:41.279724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:41.281052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:41.287614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 505/1030] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Jul 25 08:33:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:41.288301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fa585-6185-4659-8eef-207d288d3954 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:41.397181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fa585-6185-4659-8eef-207d288d3954 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:41.430799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b073959-69b9-47c4-8349-8381e5742b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:41.431471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b073959-69b9-47c4-8349-8381e5742b83 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:41.431751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b073959-69b9-47c4-8349-8381e5742b83 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:41.432024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b073959-69b9-47c4-8349-8381e5742b83 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:41.432796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b073959-69b9-47c4-8349-8381e5742b83 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:41.433104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b073959-69b9-47c4-8349-8381e5742b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:41.434458 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b073959-69b9-47c4-8349-8381e5742b83 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:41.442713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 526/1031] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Jul 25 08:33:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:41.448104 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd95ea-ca8e-4ab4-be96-4615a3966c6f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:41.533930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:41.549949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd95ea-ca8e-4ab4-be96-4615a3966c6f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:41.583410 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb86fc5-c045-4831-ad4e-5c661763c9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:41.584212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb86fc5-c045-4831-ad4e-5c661763c9ab None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:41.584494 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb86fc5-c045-4831-ad4e-5c661763c9ab None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:41.584797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb86fc5-c045-4831-ad4e-5c661763c9ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:41.595967 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb86fc5-c045-4831-ad4e-5c661763c9ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:41.597432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb86fc5-c045-4831-ad4e-5c661763c9ab None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-0cb86fc5-c045-4831-ad4e-5c661763c9ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:41.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:41.615079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:41.615998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:41.615998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:41.615998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:41.615998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:41.615998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 506/1032] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:41.622026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b073959-69b9-47c4-8349-8381e5742b83 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:41.651785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:41.726936 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b073959-69b9-47c4-8349-8381e5742b83 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:41.760487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2729e-6e43-454f-a0a7-1e0eb8728139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:41.761324 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2729e-6e43-454f-a0a7-1e0eb8728139 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:41.761616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2729e-6e43-454f-a0a7-1e0eb8728139 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:41.761895 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2729e-6e43-454f-a0a7-1e0eb8728139 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:41.762621 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2729e-6e43-454f-a0a7-1e0eb8728139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:41.762928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2729e-6e43-454f-a0a7-1e0eb8728139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:41.764279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2729e-6e43-454f-a0a7-1e0eb8728139 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:41.780737 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 527/1033] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:41.782431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb86fc5-c045-4831-ad4e-5c661763c9ab None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:41.886260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb86fc5-c045-4831-ad4e-5c661763c9ab None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:41.921747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2642f92-34da-41fe-ba7e-b0899a4786ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:41.922468 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2642f92-34da-41fe-ba7e-b0899a4786ae None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:41.922785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2642f92-34da-41fe-ba7e-b0899a4786ae None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:41.923063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2642f92-34da-41fe-ba7e-b0899a4786ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:41.930831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2642f92-34da-41fe-ba7e-b0899a4786ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:41.932272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2642f92-34da-41fe-ba7e-b0899a4786ae None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:41.933980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-e2642f92-34da-41fe-ba7e-b0899a4786ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:41.948236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:41.949417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:41.950448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:41.950448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:41.950448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:41.950448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:41.950448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:41.950448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:41.950448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:41.950448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:41.950448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:33:41.950448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:41.951551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 507/1034] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Jul 25 08:33:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:41.955925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2729e-6e43-454f-a0a7-1e0eb8728139 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:42.044009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:42.060319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2729e-6e43-454f-a0a7-1e0eb8728139 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:42.095066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef10bc-9c34-4c99-83d5-1a30cfa254bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:42.095741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef10bc-9c34-4c99-83d5-1a30cfa254bd None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:42.096027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef10bc-9c34-4c99-83d5-1a30cfa254bd None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:42.096307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef10bc-9c34-4c99-83d5-1a30cfa254bd None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:42.097173 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef10bc-9c34-4c99-83d5-1a30cfa254bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:42.097488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef10bc-9c34-4c99-83d5-1a30cfa254bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:42.098906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef10bc-9c34-4c99-83d5-1a30cfa254bd None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:42.120794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 528/1035] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Jul 25 08:33:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:42.122038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2642f92-34da-41fe-ba7e-b0899a4786ae None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:42.225004 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2642f92-34da-41fe-ba7e-b0899a4786ae None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:42.258858 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e24b4-fcb3-4ea2-ad80-c2f819bafe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:42.259519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e24b4-fcb3-4ea2-ad80-c2f819bafe42 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:42.259806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e24b4-fcb3-4ea2-ad80-c2f819bafe42 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:42.260087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e24b4-fcb3-4ea2-ad80-c2f819bafe42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:42.331463 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:42.335362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e24b4-fcb3-4ea2-ad80-c2f819bafe42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:42.336850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e24b4-fcb3-4ea2-ad80-c2f819bafe42 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:42.442894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 508/1036] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:42 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:33:42.444723 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef10bc-9c34-4c99-83d5-1a30cfa254bd None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:42.549206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef10bc-9c34-4c99-83d5-1a30cfa254bd None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:42.582917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcbeab-a22b-444c-bf21-b32d82262fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:42.583711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcbeab-a22b-444c-bf21-b32d82262fda None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:42.583994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcbeab-a22b-444c-bf21-b32d82262fda None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:42.584263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcbeab-a22b-444c-bf21-b32d82262fda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:42.594467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddcbeab-a22b-444c-bf21-b32d82262fda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:42.596433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddcbeab-a22b-444c-bf21-b32d82262fda None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:42.599267 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-3ddcbeab-a22b-444c-bf21-b32d82262fda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:42.615466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:42.616391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:42.617306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:42.617306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:42.617306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:42.617306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:42.620954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 529/1037] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:42.622446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e24b4-fcb3-4ea2-ad80-c2f819bafe42 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:42.719372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:42.724434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e24b4-fcb3-4ea2-ad80-c2f819bafe42 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:42.758407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3b9cb-fc06-4e75-b711-7ac7c2c57fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:42.759068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3b9cb-fc06-4e75-b711-7ac7c2c57fbd None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:42.759348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3b9cb-fc06-4e75-b711-7ac7c2c57fbd None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:42.759621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3b9cb-fc06-4e75-b711-7ac7c2c57fbd None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:42.760341 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3b9cb-fc06-4e75-b711-7ac7c2c57fbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:42.760672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3b9cb-fc06-4e75-b711-7ac7c2c57fbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:42.762032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3b9cb-fc06-4e75-b711-7ac7c2c57fbd None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:42.778023 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 509/1038] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:42.779338 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcbeab-a22b-444c-bf21-b32d82262fda None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:42.883368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcbeab-a22b-444c-bf21-b32d82262fda None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:42.917419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de85d05-fa62-4c71-b830-def0a5bbb3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:42.918208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de85d05-fa62-4c71-b830-def0a5bbb3fe None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:42.918489 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de85d05-fa62-4c71-b830-def0a5bbb3fe None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:42.918763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de85d05-fa62-4c71-b830-def0a5bbb3fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:42.924102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27146e-aada-4b90-9b60-231a8878debe None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:42.924391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27146e-aada-4b90-9b60-231a8878debe None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:42.924706 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27146e-aada-4b90-9b60-231a8878debe None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:42.925929 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de85d05-fa62-4c71-b830-def0a5bbb3fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:42.926331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 510/1039] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:42.927367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de85d05-fa62-4c71-b830-def0a5bbb3fe None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:42.931038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96349b-cb62-4c08-8164-a2f4834fb76d None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:42.931327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96349b-cb62-4c08-8164-a2f4834fb76d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:42.931603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96349b-cb62-4c08-8164-a2f4834fb76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-7de85d05-fa62-4c71-b830-def0a5bbb3fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:42.942964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:42.943842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:42.945175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:42.945175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:42.945175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:42.945175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:42.945175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:42.945175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:42.945175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:42.945175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:42.949075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 530/1040] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:42.950114 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-1c96349b-cb62-4c08-8164-a2f4834fb76d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:42.950264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de85d05-fa62-4c71-b830-def0a5bbb3fe None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:42.951284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1c96349b-cb62-4c08-8164-a2f4834fb76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:43.011911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:43.028613 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 511/1041] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:43.050063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96349b-cb62-4c08-8164-a2f4834fb76d None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:43.058919 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de85d05-fa62-4c71-b830-def0a5bbb3fe None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:43.093735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c64326-c392-4282-9f14-f588bada80a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:43.094404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c64326-c392-4282-9f14-f588bada80a4 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:43.095320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c64326-c392-4282-9f14-f588bada80a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:43.095597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c64326-c392-4282-9f14-f588bada80a4 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:43.096387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c64326-c392-4282-9f14-f588bada80a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:43.096723 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c64326-c392-4282-9f14-f588bada80a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:43.098213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c64326-c392-4282-9f14-f588bada80a4 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:43.110527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 531/1042] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:43.116343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c64326-c392-4282-9f14-f588bada80a4 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:43.145155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:43.161271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96349b-cb62-4c08-8164-a2f4834fb76d None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:43.196516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394286a-3940-4329-95e0-5b4968e2ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:43.197346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394286a-3940-4329-95e0-5b4968e2ade5 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:43.197629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394286a-3940-4329-95e0-5b4968e2ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:43.197960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394286a-3940-4329-95e0-5b4968e2ade5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:43.199153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394286a-3940-4329-95e0-5b4968e2ade5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:43.200604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394286a-3940-4329-95e0-5b4968e2ade5 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:43.204722 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 512/1043] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:43.208544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394286a-3940-4329-95e0-5b4968e2ade5 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:43.221993 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c64326-c392-4282-9f14-f588bada80a4 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:43.238256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:43.256748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95039bc2-1b04-49fe-a9ff-cdcbf15a4fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:43.257422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95039bc2-1b04-49fe-a9ff-cdcbf15a4fea None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:43.257688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95039bc2-1b04-49fe-a9ff-cdcbf15a4fea None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:43.257964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95039bc2-1b04-49fe-a9ff-cdcbf15a4fea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:43.258791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95039bc2-1b04-49fe-a9ff-cdcbf15a4fea None admin] RBAC: Adding query filter params (role.id=1ac8ff72ed404116b52f57f20668457d, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=8b476d6b82ae464b81fc8498dc0e68c7) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:43.259100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95039bc2-1b04-49fe-a9ff-cdcbf15a4fea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:43.260506 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95039bc2-1b04-49fe-a9ff-cdcbf15a4fea None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:43.265935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 532/1044] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:43 2022] GET /identity/v3/role_assignments?user.id=8b476d6b82ae464b81fc8498dc0e68c7&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=1ac8ff72ed404116b52f57f20668457d => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:43.267769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5e182-251f-4382-9b89-675a03e61a86 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:43.268055 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5e182-251f-4382-9b89-675a03e61a86 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:43.268332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5e182-251f-4382-9b89-675a03e61a86 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:43.269790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 533/1045] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:43.276653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b091a69-14a7-48fb-8db9-1a6805a9cb9b None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:43.276944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b091a69-14a7-48fb-8db9-1a6805a9cb9b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:43.277223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b091a69-14a7-48fb-8db9-1a6805a9cb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:43.300206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-3b091a69-14a7-48fb-8db9-1a6805a9cb9b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:43.301855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3b091a69-14a7-48fb-8db9-1a6805a9cb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:43.325659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394286a-3940-4329-95e0-5b4968e2ade5 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:43.366430 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab8ad-3c1b-4fd2-9b81-b73bde87635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:43.367242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab8ad-3c1b-4fd2-9b81-b73bde87635e None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:43.367528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab8ad-3c1b-4fd2-9b81-b73bde87635e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:43.367809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab8ad-3c1b-4fd2-9b81-b73bde87635e None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:43.368539 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eab8ad-3c1b-4fd2-9b81-b73bde87635e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:43.370256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eab8ad-3c1b-4fd2-9b81-b73bde87635e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:43.377913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 534/1046] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:43.404689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 513/1047] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:33:43 2022] POST /identity/v3/users => generated 285 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:33:43.404898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b091a69-14a7-48fb-8db9-1a6805a9cb9b None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:43.510069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b091a69-14a7-48fb-8db9-1a6805a9cb9b None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:43.522121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:43.550572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9adc9-8991-45a1-a6d5-9f7378de139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:43.551706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9adc9-8991-45a1-a6d5-9f7378de139b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:43.552148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9adc9-8991-45a1-a6d5-9f7378de139b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:43.552421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9adc9-8991-45a1-a6d5-9f7378de139b None admin] PATH_INFO: `/v3/services/image` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:43.553115 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9adc9-8991-45a1-a6d5-9f7378de139b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:43.554477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9adc9-8991-45a1-a6d5-9f7378de139b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-a4f9adc9-8991-45a1-a6d5-9f7378de139b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:43.566860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:43.567873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:43.568821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 535/1048] 10.222.0.52 () {56 vars in 1105 bytes} [Mon Jul 25 08:33:43 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:43.576162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab8ad-3c1b-4fd2-9b81-b73bde87635e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:43.592463 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:43.682740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab8ad-3c1b-4fd2-9b81-b73bde87635e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:43.718389 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f58a86-22d8-44a0-83c5-7eee0f8317b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:43.719075 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f58a86-22d8-44a0-83c5-7eee0f8317b4 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:43.719357 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f58a86-22d8-44a0-83c5-7eee0f8317b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:43.720179 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f58a86-22d8-44a0-83c5-7eee0f8317b4 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:43.720934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f58a86-22d8-44a0-83c5-7eee0f8317b4 None admin] RBAC: Adding query filter params (name=image) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:43.721246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f58a86-22d8-44a0-83c5-7eee0f8317b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:43.722623 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f58a86-22d8-44a0-83c5-7eee0f8317b4 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:43.728164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 514/1049] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:43.734148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9adc9-8991-45a1-a6d5-9f7378de139b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:43.838152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9adc9-8991-45a1-a6d5-9f7378de139b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:43.872163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2dc11-6d12-4b8a-a377-227c15cfc197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:43.872860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2dc11-6d12-4b8a-a377-227c15cfc197 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:43.873146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2dc11-6d12-4b8a-a377-227c15cfc197 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:43.873421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2dc11-6d12-4b8a-a377-227c15cfc197 None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:43.874130 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2dc11-6d12-4b8a-a377-227c15cfc197 None admin] RBAC: Adding query filter params (type=image) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:43.874434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2dc11-6d12-4b8a-a377-227c15cfc197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:43.875857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2dc11-6d12-4b8a-a377-227c15cfc197 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:43.885519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 536/1050] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:43.887052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f58a86-22d8-44a0-83c5-7eee0f8317b4 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:43.967058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:43.991433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f58a86-22d8-44a0-83c5-7eee0f8317b4 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:44.025519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf1537-f53e-49de-836f-218b2c17e3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:44.026448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf1537-f53e-49de-836f-218b2c17e3b7 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:44.026732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf1537-f53e-49de-836f-218b2c17e3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:44.027013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf1537-f53e-49de-836f-218b2c17e3b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:44.027763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf1537-f53e-49de-836f-218b2c17e3b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:44.029167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf1537-f53e-49de-836f-218b2c17e3b7 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:44.060624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 515/1051] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:33:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:45.261105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627051cb-46f2-4842-974d-c55f31ff2d74 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:45.261631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627051cb-46f2-4842-974d-c55f31ff2d74 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:45.261710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627051cb-46f2-4842-974d-c55f31ff2d74 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:45.263207 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 537/1052] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:45.268824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002da47f-04f2-4855-aa4f-2bf1379f8ec1 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:45.269260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002da47f-04f2-4855-aa4f-2bf1379f8ec1 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:45.269406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002da47f-04f2-4855-aa4f-2bf1379f8ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:45.291232 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-002da47f-04f2-4855-aa4f-2bf1379f8ec1 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:45.291232 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-002da47f-04f2-4855-aa4f-2bf1379f8ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:45.368607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 516/1053] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:45.389747 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627051cb-46f2-4842-974d-c55f31ff2d74 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:45.433075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:45.459082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2056e-44a8-4e23-a22a-34bd8803fb09 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:45.459691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2056e-44a8-4e23-a22a-34bd8803fb09 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:45.460197 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2056e-44a8-4e23-a22a-34bd8803fb09 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:45.462589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 517/1054] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:45.467415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc3974-a802-487b-8dfc-e9001276eccd None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:45.467929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc3974-a802-487b-8dfc-e9001276eccd None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:45.468411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc3974-a802-487b-8dfc-e9001276eccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:45.489875 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-fcfc3974-a802-487b-8dfc-e9001276eccd None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:45.491049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-fcfc3974-a802-487b-8dfc-e9001276eccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:45.499203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627051cb-46f2-4842-974d-c55f31ff2d74 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:45.538049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e125f-d30f-4c2b-a315-b3f1c15a42f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:45.539245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e125f-d30f-4c2b-a315-b3f1c15a42f4 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:45.539726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e125f-d30f-4c2b-a315-b3f1c15a42f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:45.540261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e125f-d30f-4c2b-a315-b3f1c15a42f4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:45.546374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:45.553972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e125f-d30f-4c2b-a315-b3f1c15a42f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:45.562727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 518/1055] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-158e125f-d30f-4c2b-a315-b3f1c15a42f4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:45.562872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:45.564449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:45.565601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:45.565601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:45.565601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:45.565601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:45.565601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:45.565601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:45.565601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:45.565601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:45.565601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:45.565601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:45.565601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 08:33:45.565601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:45.567870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 538/1056] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Jul 25 08:33:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:45.573439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc3974-a802-487b-8dfc-e9001276eccd None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:45.586491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e125f-d30f-4c2b-a315-b3f1c15a42f4 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:45.683139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc3974-a802-487b-8dfc-e9001276eccd None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:45.690439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:45.694898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e125f-d30f-4c2b-a315-b3f1c15a42f4 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:45.721582 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe86f40-61a2-47f9-a182-748426d2cf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:45.722268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe86f40-61a2-47f9-a182-748426d2cf93 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:45.722555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe86f40-61a2-47f9-a182-748426d2cf93 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:45.722955 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe86f40-61a2-47f9-a182-748426d2cf93 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:45.723832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe86f40-61a2-47f9-a182-748426d2cf93 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:45.724143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe86f40-61a2-47f9-a182-748426d2cf93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:45.725697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe86f40-61a2-47f9-a182-748426d2cf93 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:45.731949 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60711d7f-a9ac-41f4-acb2-2706b078ef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:45.732783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60711d7f-a9ac-41f4-acb2-2706b078ef63 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:45.733072 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60711d7f-a9ac-41f4-acb2-2706b078ef63 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:45.733352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60711d7f-a9ac-41f4-acb2-2706b078ef63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:45.738789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 519/1057] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Jul 25 08:33:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:45.740336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe86f40-61a2-47f9-a182-748426d2cf93 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:45.749695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60711d7f-a9ac-41f4-acb2-2706b078ef63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-60711d7f-a9ac-41f4-acb2-2706b078ef63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:45.757955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:45.758881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:45.759732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 539/1058] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:33:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:45.765103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60711d7f-a9ac-41f4-acb2-2706b078ef63 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:45.805854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:45.858949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe86f40-61a2-47f9-a182-748426d2cf93 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:45.885003 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60711d7f-a9ac-41f4-acb2-2706b078ef63 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:45.894274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c204fe-28b6-47a0-be96-e971d3d62605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:45.895090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c204fe-28b6-47a0-be96-e971d3d62605 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:45.895383 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c204fe-28b6-47a0-be96-e971d3d62605 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:45.895664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c204fe-28b6-47a0-be96-e971d3d62605 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:45.907764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c204fe-28b6-47a0-be96-e971d3d62605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:45.909251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c204fe-28b6-47a0-be96-e971d3d62605 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:45.923225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa98cd-eff4-47aa-bcb5-6f421a0ae9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:45.924117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa98cd-eff4-47aa-bcb5-6f421a0ae9cc None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:45.924632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa98cd-eff4-47aa-bcb5-6f421a0ae9cc None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:45.925084 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa98cd-eff4-47aa-bcb5-6f421a0ae9cc None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:45.926000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa98cd-eff4-47aa-bcb5-6f421a0ae9cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:45.926480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa98cd-eff4-47aa-bcb5-6f421a0ae9cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-e5c204fe-28b6-47a0-be96-e971d3d62605 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:45.926585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:45.927405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:45.928297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:45.928297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:45.928297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:45.928297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:45.928297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 520/1059] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:45.930141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa98cd-eff4-47aa-bcb5-6f421a0ae9cc None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:45.933381 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c204fe-28b6-47a0-be96-e971d3d62605 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:45.945353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 540/1060] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:33:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:45.946263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa98cd-eff4-47aa-bcb5-6f421a0ae9cc None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:45.969847 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:46.042880 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c204fe-28b6-47a0-be96-e971d3d62605 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:46.044731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:46.058578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa98cd-eff4-47aa-bcb5-6f421a0ae9cc None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:46.083570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baffbde-8b14-4681-a685-d0e875bae34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:46.084245 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baffbde-8b14-4681-a685-d0e875bae34f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:46.084530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baffbde-8b14-4681-a685-d0e875bae34f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:46.084847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baffbde-8b14-4681-a685-d0e875bae34f None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:46.085580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baffbde-8b14-4681-a685-d0e875bae34f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:46.085894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baffbde-8b14-4681-a685-d0e875bae34f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:46.087263 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baffbde-8b14-4681-a685-d0e875bae34f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:46.092520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3621514-81a3-4029-bca3-02d9df2f0af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:46.093425 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3621514-81a3-4029-bca3-02d9df2f0af0 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:46.093710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3621514-81a3-4029-bca3-02d9df2f0af0 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:46.093988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3621514-81a3-4029-bca3-02d9df2f0af0 None admin] PATH_INFO: `/v3/users/36541892a93347d5801f6d5229e9dbab` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:46.103249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 521/1061] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:46.103249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745d443-fd88-4684-84b3-f9d96c90b852 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:46.103249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745d443-fd88-4684-84b3-f9d96c90b852 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:46.103249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745d443-fd88-4684-84b3-f9d96c90b852 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:46.103249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 522/1062] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:46.104454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745d443-fd88-4684-84b3-f9d96c90b852 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:46.108096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3621514-81a3-4029-bca3-02d9df2f0af0 None admin] RBAC: Authorizing `identity:get_user(user_id=36541892a93347d5801f6d5229e9dbab)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:46.109567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3621514-81a3-4029-bca3-02d9df2f0af0 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:46.111939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 541/1063] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:33:45 2022] GET /identity/v3/users/36541892a93347d5801f6d5229e9dbab => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:46.113167 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663abf1-e6d0-4bba-982c-f2cad7fd1061 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:46.113459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663abf1-e6d0-4bba-982c-f2cad7fd1061 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:46.113738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663abf1-e6d0-4bba-982c-f2cad7fd1061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:46.131502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-6663abf1-e6d0-4bba-982c-f2cad7fd1061 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:46.132731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6663abf1-e6d0-4bba-982c-f2cad7fd1061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:46.198698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 542/1064] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:46.199530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663abf1-e6d0-4bba-982c-f2cad7fd1061 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:46.209668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745d443-fd88-4684-84b3-f9d96c90b852 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:46.209923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:46.245332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2474a5-4c22-43d5-b3a5-961ea25e355a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:46.246130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2474a5-4c22-43d5-b3a5-961ea25e355a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:46.246415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2474a5-4c22-43d5-b3a5-961ea25e355a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:46.246698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2474a5-4c22-43d5-b3a5-961ea25e355a None admin] PATH_INFO: `/v3/users/nova` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:46.254462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2474a5-4c22-43d5-b3a5-961ea25e355a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:46.255903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2474a5-4c22-43d5-b3a5-961ea25e355a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-1e2474a5-4c22-43d5-b3a5-961ea25e355a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:46.270031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:46.271405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:46.272365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:46.272365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:46.272365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:46.272365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:46.272365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:46.272365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:46.272365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:46.272365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:46.272365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:33:46.272365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:46.272963 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 523/1065] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Jul 25 08:33:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:46.273821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2474a5-4c22-43d5-b3a5-961ea25e355a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:46.298669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:46.299207 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663abf1-e6d0-4bba-982c-f2cad7fd1061 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:46.336134 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad22f74-7650-464c-b086-e48bfb9211df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:46.336761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad22f74-7650-464c-b086-e48bfb9211df None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:46.337051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad22f74-7650-464c-b086-e48bfb9211df None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:46.337333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad22f74-7650-464c-b086-e48bfb9211df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:46.376853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2474a5-4c22-43d5-b3a5-961ea25e355a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:46.403733 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad22f74-7650-464c-b086-e48bfb9211df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:46.405243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad22f74-7650-464c-b086-e48bfb9211df None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:46.409907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c7c31-c1b6-4f81-bb1b-dddbda378bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:46.410549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c7c31-c1b6-4f81-bb1b-dddbda378bc4 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:46.410830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c7c31-c1b6-4f81-bb1b-dddbda378bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:46.411106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c7c31-c1b6-4f81-bb1b-dddbda378bc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:46.415677 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:46.476798 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c7c31-c1b6-4f81-bb1b-dddbda378bc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:46.478253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c7c31-c1b6-4f81-bb1b-dddbda378bc4 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:46.500753 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:46.505925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 543/1066] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:33:46.506753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad22f74-7650-464c-b086-e48bfb9211df None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:46.578024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 524/1067] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:33:46.578916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c7c31-c1b6-4f81-bb1b-dddbda378bc4 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:46.604634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad22f74-7650-464c-b086-e48bfb9211df None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:46.614243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:46.638954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca749b8f-ae14-4a02-899e-782b98248bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:46.639620 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca749b8f-ae14-4a02-899e-782b98248bb3 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:46.639905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca749b8f-ae14-4a02-899e-782b98248bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:46.640182 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca749b8f-ae14-4a02-899e-782b98248bb3 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:46.641045 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca749b8f-ae14-4a02-899e-782b98248bb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:46.641880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca749b8f-ae14-4a02-899e-782b98248bb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:46.643290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca749b8f-ae14-4a02-899e-782b98248bb3 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:46.660754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 544/1068] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Jul 25 08:33:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:46.661553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca749b8f-ae14-4a02-899e-782b98248bb3 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:46.680491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c7c31-c1b6-4f81-bb1b-dddbda378bc4 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:46.702371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:46.716605 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc65e0-80f8-46ea-8e11-2567f958a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:46.717455 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc65e0-80f8-46ea-8e11-2567f958a604 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:46.717738 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc65e0-80f8-46ea-8e11-2567f958a604 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:46.718019 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc65e0-80f8-46ea-8e11-2567f958a604 None admin] PATH_INFO: `/v3/projects/55d8e4202835461682a7999ca8a66e2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:46.719200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc65e0-80f8-46ea-8e11-2567f958a604 None admin] RBAC: Authorizing `identity:get_project(project_id=55d8e4202835461682a7999ca8a66e2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:46.721203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc65e0-80f8-46ea-8e11-2567f958a604 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:46.722903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 525/1069] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:33:46 2022] GET /identity/v3/projects/55d8e4202835461682a7999ca8a66e2a => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:46.723655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc65e0-80f8-46ea-8e11-2567f958a604 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:46.762708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca749b8f-ae14-4a02-899e-782b98248bb3 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:46.797024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff0f72-4c9d-4a8c-bed3-30b280de3ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:46.797815 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff0f72-4c9d-4a8c-bed3-30b280de3ee1 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:46.798101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff0f72-4c9d-4a8c-bed3-30b280de3ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:46.798379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff0f72-4c9d-4a8c-bed3-30b280de3ee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:46.808464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff0f72-4c9d-4a8c-bed3-30b280de3ee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:46.809939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff0f72-4c9d-4a8c-bed3-30b280de3ee1 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:46.819555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc65e0-80f8-46ea-8e11-2567f958a604 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-faff0f72-4c9d-4a8c-bed3-30b280de3ee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:46.827414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:46.828308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:46.829767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:46.829767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:46.829767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:46.829767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:46.829767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 545/1070] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:46.830053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff0f72-4c9d-4a8c-bed3-30b280de3ee1 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:46.848757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:46.855473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580c91c-909c-435f-9193-98a721dba016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:46.856202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580c91c-909c-435f-9193-98a721dba016 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:46.856478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580c91c-909c-435f-9193-98a721dba016 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:46.856786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580c91c-909c-435f-9193-98a721dba016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:46.867739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580c91c-909c-435f-9193-98a721dba016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:46.869205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580c91c-909c-435f-9193-98a721dba016 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-d580c91c-909c-435f-9193-98a721dba016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:46.885721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:46.886675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:46.887567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:46.887567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:46.887567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:46.887567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:46.887567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 526/1071] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:46.887844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580c91c-909c-435f-9193-98a721dba016 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:46.929725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:46.931597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff0f72-4c9d-4a8c-bed3-30b280de3ee1 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:46.965199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4520127-84d8-445e-810f-874a593bddbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:46.965844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4520127-84d8-445e-810f-874a593bddbe None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:46.966135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4520127-84d8-445e-810f-874a593bddbe None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:46.966406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4520127-84d8-445e-810f-874a593bddbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:46.967233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4520127-84d8-445e-810f-874a593bddbe None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=55d8e4202835461682a7999ca8a66e2a, user.id=36541892a93347d5801f6d5229e9dbab) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:46.967539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4520127-84d8-445e-810f-874a593bddbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:46.968993 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4520127-84d8-445e-810f-874a593bddbe None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:46.974832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 546/1072] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:46 2022] GET /identity/v3/role_assignments?user.id=36541892a93347d5801f6d5229e9dbab&scope.project.id=55d8e4202835461682a7999ca8a66e2a&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:46.975595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4520127-84d8-445e-810f-874a593bddbe None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:46.991445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580c91c-909c-435f-9193-98a721dba016 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:47.032767 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e11cc-8e4a-4810-8df8-ed65f87c1e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:47.033551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e11cc-8e4a-4810-8df8-ed65f87c1e3b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:47.033836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e11cc-8e4a-4810-8df8-ed65f87c1e3b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:47.034106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e11cc-8e4a-4810-8df8-ed65f87c1e3b None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:47.035372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e11cc-8e4a-4810-8df8-ed65f87c1e3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:47.035372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e11cc-8e4a-4810-8df8-ed65f87c1e3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:47.036533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e11cc-8e4a-4810-8df8-ed65f87c1e3b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:47.051848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 527/1073] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:47.096255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:47.119347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4520127-84d8-445e-810f-874a593bddbe None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:47.162901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df1eaa-228e-4239-a8f1-1932dfb2e492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:47.163581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df1eaa-228e-4239-a8f1-1932dfb2e492 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:47.163869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df1eaa-228e-4239-a8f1-1932dfb2e492 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:47.164150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df1eaa-228e-4239-a8f1-1932dfb2e492 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:47.164932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df1eaa-228e-4239-a8f1-1932dfb2e492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:47.165251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df1eaa-228e-4239-a8f1-1932dfb2e492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:47.166650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df1eaa-228e-4239-a8f1-1932dfb2e492 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:47.185281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 547/1074] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:47.186131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e11cc-8e4a-4810-8df8-ed65f87c1e3b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:47.295319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e11cc-8e4a-4810-8df8-ed65f87c1e3b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:47.297058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:47.336030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61dafd8-5ce5-43c3-bd2b-63f1b15c5069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:47.337155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dafd8-5ce5-43c3-bd2b-63f1b15c5069 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:47.337619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dafd8-5ce5-43c3-bd2b-63f1b15c5069 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:47.338100 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dafd8-5ce5-43c3-bd2b-63f1b15c5069 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:47.346909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61dafd8-5ce5-43c3-bd2b-63f1b15c5069 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:47.347099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61dafd8-5ce5-43c3-bd2b-63f1b15c5069 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-a61dafd8-5ce5-43c3-bd2b-63f1b15c5069 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:47.360976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:47.362067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:47.363168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:47.363168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:47.363168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:47.363168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:47.363168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:47.363168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:47.363168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:47.363168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:47.363168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 528/1075] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:47.369912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df1eaa-228e-4239-a8f1-1932dfb2e492 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:47.467961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df1eaa-228e-4239-a8f1-1932dfb2e492 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:47.501527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd59c5-8872-4a20-8dc8-88d27b1b52f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:47.502178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd59c5-8872-4a20-8dc8-88d27b1b52f6 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:47.502466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd59c5-8872-4a20-8dc8-88d27b1b52f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:47.502740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd59c5-8872-4a20-8dc8-88d27b1b52f6 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:47.503517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd59c5-8872-4a20-8dc8-88d27b1b52f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:47.503821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd59c5-8872-4a20-8dc8-88d27b1b52f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:47.505257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd59c5-8872-4a20-8dc8-88d27b1b52f6 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:47.516472 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 548/1076] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:47.522461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61dafd8-5ce5-43c3-bd2b-63f1b15c5069 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:47.620521 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61dafd8-5ce5-43c3-bd2b-63f1b15c5069 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:47.653567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe700f7-feae-4510-ab13-d6447418e8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:47.654465 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe700f7-feae-4510-ab13-d6447418e8d9 None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:47.654747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe700f7-feae-4510-ab13-d6447418e8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:47.655022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe700f7-feae-4510-ab13-d6447418e8d9 None admin] PATH_INFO: `/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/8b476d6b82ae464b81fc8498dc0e68c7/roles/1ac8ff72ed404116b52f57f20668457d` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:47.657300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe700f7-feae-4510-ab13-d6447418e8d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ee27c4da77c47428e02fd65e890d256, user_id=8b476d6b82ae464b81fc8498dc0e68c7, role_id=1ac8ff72ed404116b52f57f20668457d)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:47.658734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe700f7-feae-4510-ab13-d6447418e8d9 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:47.671351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 529/1077] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:33:47 2022] PUT /identity/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/8b476d6b82ae464b81fc8498dc0e68c7/roles/1ac8ff72ed404116b52f57f20668457d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:33:48.950687 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a0a5d-7aac-4824-a01b-7d7524282e7a None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:48.951243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a0a5d-7aac-4824-a01b-7d7524282e7a None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:48.951243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a0a5d-7aac-4824-a01b-7d7524282e7a None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:48.952784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 549/1078] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:48.957802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0ec87-d2b2-4919-b0e9-06f3dee9d8a0 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:48.958237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0ec87-d2b2-4919-b0e9-06f3dee9d8a0 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:48.958458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0ec87-d2b2-4919-b0e9-06f3dee9d8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:48.976177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-d7c0ec87-d2b2-4919-b0e9-06f3dee9d8a0 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:48.977385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d7c0ec87-d2b2-4919-b0e9-06f3dee9d8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:48.993932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:49.045959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 530/1079] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:49.068526 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a0a5d-7aac-4824-a01b-7d7524282e7a None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:49.071426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:49.168264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a0a5d-7aac-4824-a01b-7d7524282e7a None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:49.201868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be377c9-b613-4c92-a731-55e729f62d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:49.202595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be377c9-b613-4c92-a731-55e729f62d93 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:49.202881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be377c9-b613-4c92-a731-55e729f62d93 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:49.203159 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be377c9-b613-4c92-a731-55e729f62d93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:49.213192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be377c9-b613-4c92-a731-55e729f62d93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-1be377c9-b613-4c92-a731-55e729f62d93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:49.220763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:49.221872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:49.222713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 550/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:33:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:49.227871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0ec87-d2b2-4919-b0e9-06f3dee9d8a0 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:49.333930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0ec87-d2b2-4919-b0e9-06f3dee9d8a0 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:49.367268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd831f0-0978-41c4-b09e-73ee537fa2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:49.367937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd831f0-0978-41c4-b09e-73ee537fa2de None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:49.368221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd831f0-0978-41c4-b09e-73ee537fa2de None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:49.368497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd831f0-0978-41c4-b09e-73ee537fa2de None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:49.369274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd831f0-0978-41c4-b09e-73ee537fa2de None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:49.369580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd831f0-0978-41c4-b09e-73ee537fa2de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:49.370934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd831f0-0978-41c4-b09e-73ee537fa2de None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:49.378981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 531/1081] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:33:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:49.384687 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be377c9-b613-4c92-a731-55e729f62d93 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:49.444703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:49.482926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be377c9-b613-4c92-a731-55e729f62d93 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:49.515615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fc378-927e-499e-bb70-27c0c6830a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:49.516511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fc378-927e-499e-bb70-27c0c6830a05 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:49.516822 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fc378-927e-499e-bb70-27c0c6830a05 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:49.517099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fc378-927e-499e-bb70-27c0c6830a05 None admin] PATH_INFO: `/v3/users/36541892a93347d5801f6d5229e9dbab` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:49.518283 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fc378-927e-499e-bb70-27c0c6830a05 None admin] RBAC: Authorizing `identity:get_user(user_id=36541892a93347d5801f6d5229e9dbab)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:49.519712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fc378-927e-499e-bb70-27c0c6830a05 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:49.521387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 551/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:33:49 2022] GET /identity/v3/users/36541892a93347d5801f6d5229e9dbab => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:49.527315 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd831f0-0978-41c4-b09e-73ee537fa2de None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:49.543588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:49.619502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d41d5-2eef-42f9-817b-31ed5288be97 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:49.619785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d41d5-2eef-42f9-817b-31ed5288be97 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:49.620065 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d41d5-2eef-42f9-817b-31ed5288be97 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:49.621484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 552/1083] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:49.626304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d1b21-3563-499e-9935-737d9d5e6f4a None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:49.626552 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd831f0-0978-41c4-b09e-73ee537fa2de None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:49.626657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d1b21-3563-499e-9935-737d9d5e6f4a None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:49.626850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d1b21-3563-499e-9935-737d9d5e6f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:49.645490 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-3c8d1b21-3563-499e-9935-737d9d5e6f4a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:49.646678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3c8d1b21-3563-499e-9935-737d9d5e6f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:49.661524 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f9de6-f45f-4dd1-b5a2-8d09489e5ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:49.662380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f9de6-f45f-4dd1-b5a2-8d09489e5ab7 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:49.662662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f9de6-f45f-4dd1-b5a2-8d09489e5ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:49.662941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f9de6-f45f-4dd1-b5a2-8d09489e5ab7 None admin] PATH_INFO: `/v3/projects/55d8e4202835461682a7999ca8a66e2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:49.664104 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f9de6-f45f-4dd1-b5a2-8d09489e5ab7 None admin] RBAC: Authorizing `identity:get_project(project_id=55d8e4202835461682a7999ca8a66e2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:49.665552 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f9de6-f45f-4dd1-b5a2-8d09489e5ab7 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:49.667261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 532/1084] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:33:49 2022] GET /identity/v3/projects/55d8e4202835461682a7999ca8a66e2a => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:33:49.673053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f9de6-f45f-4dd1-b5a2-8d09489e5ab7 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:49.714862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 553/1085] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:49.738057 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d1b21-3563-499e-9935-737d9d5e6f4a None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:49.752536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:49.771862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f9de6-f45f-4dd1-b5a2-8d09489e5ab7 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:49.795575 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:49.805434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184ea32-1539-4ffc-b654-493f934b551b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:49.806327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184ea32-1539-4ffc-b654-493f934b551b None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:49.806611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184ea32-1539-4ffc-b654-493f934b551b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:49.806886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184ea32-1539-4ffc-b654-493f934b551b None admin] PATH_INFO: `/v3/projects/55d8e4202835461682a7999ca8a66e2a/users/36541892a93347d5801f6d5229e9dbab/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:49.808916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184ea32-1539-4ffc-b654-493f934b551b None admin] RBAC: Authorizing `identity:create_grant(project_id=55d8e4202835461682a7999ca8a66e2a, user_id=36541892a93347d5801f6d5229e9dbab, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:49.810380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184ea32-1539-4ffc-b654-493f934b551b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:49.822546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 533/1086] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:33:49 2022] PUT /identity/v3/projects/55d8e4202835461682a7999ca8a66e2a/users/36541892a93347d5801f6d5229e9dbab/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:33:49.837091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d1b21-3563-499e-9935-737d9d5e6f4a None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:49.874015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85242f4-5ee9-42ef-b7eb-5b7752c01c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:49.874867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85242f4-5ee9-42ef-b7eb-5b7752c01c5d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:49.875151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85242f4-5ee9-42ef-b7eb-5b7752c01c5d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:49.875478 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85242f4-5ee9-42ef-b7eb-5b7752c01c5d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:49.886982 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85242f4-5ee9-42ef-b7eb-5b7752c01c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-f85242f4-5ee9-42ef-b7eb-5b7752c01c5d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:49.896015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:49.897202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:49.898317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 554/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Jul 25 08:33:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:49.903839 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184ea32-1539-4ffc-b654-493f934b551b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:50.008757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184ea32-1539-4ffc-b654-493f934b551b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:50.042240 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb2f64-8fcc-4ebe-8d53-7f53271796d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:50.042915 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb2f64-8fcc-4ebe-8d53-7f53271796d0 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:50.043200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb2f64-8fcc-4ebe-8d53-7f53271796d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:50.043474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb2f64-8fcc-4ebe-8d53-7f53271796d0 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:50.044219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb2f64-8fcc-4ebe-8d53-7f53271796d0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:50.044524 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb2f64-8fcc-4ebe-8d53-7f53271796d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:50.045940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb2f64-8fcc-4ebe-8d53-7f53271796d0 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:50.053975 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 534/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Jul 25 08:33:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:50.059920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85242f4-5ee9-42ef-b7eb-5b7752c01c5d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:50.158983 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85242f4-5ee9-42ef-b7eb-5b7752c01c5d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:50.178215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:50.193026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe8303-04f8-41bc-9c92-c6619396c98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:50.193820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe8303-04f8-41bc-9c92-c6619396c98c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:50.194106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe8303-04f8-41bc-9c92-c6619396c98c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:50.194384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe8303-04f8-41bc-9c92-c6619396c98c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:50.205447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe8303-04f8-41bc-9c92-c6619396c98c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:50.206883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe8303-04f8-41bc-9c92-c6619396c98c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-9bfe8303-04f8-41bc-9c92-c6619396c98c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:50.223711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:50.224799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:50.230560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:50.230560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:50.230560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:50.230560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:50.230560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 555/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:50.230839 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb2f64-8fcc-4ebe-8d53-7f53271796d0 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:50.234496 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:50.334977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb2f64-8fcc-4ebe-8d53-7f53271796d0 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:50.367023 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de0e5c-cb1d-4138-96c8-175b363b644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:50.367690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de0e5c-cb1d-4138-96c8-175b363b644a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:50.367977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de0e5c-cb1d-4138-96c8-175b363b644a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:50.368258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de0e5c-cb1d-4138-96c8-175b363b644a None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:50.369012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de0e5c-cb1d-4138-96c8-175b363b644a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:50.369321 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de0e5c-cb1d-4138-96c8-175b363b644a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:50.370690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de0e5c-cb1d-4138-96c8-175b363b644a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:50.382577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 535/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:50.388040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe8303-04f8-41bc-9c92-c6619396c98c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:50.485740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe8303-04f8-41bc-9c92-c6619396c98c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:50.518686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14cfb0-7918-48a3-86e8-f0dff7addc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:50.519476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14cfb0-7918-48a3-86e8-f0dff7addc00 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:50.519762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14cfb0-7918-48a3-86e8-f0dff7addc00 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:50.520043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14cfb0-7918-48a3-86e8-f0dff7addc00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:50.527562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14cfb0-7918-48a3-86e8-f0dff7addc00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:50.529039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14cfb0-7918-48a3-86e8-f0dff7addc00 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-5a14cfb0-7918-48a3-86e8-f0dff7addc00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:33:50.542761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:50.543816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:50.544800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:50.544800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:50.544800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:33:50.544800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:33:50.544800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:33:50.544800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:33:50.544800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:33:50.544800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:33:50.544800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 08:33:50.544800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:33:50.548937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 556/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Jul 25 08:33:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:33:50.550161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de0e5c-cb1d-4138-96c8-175b363b644a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:50.610777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:50.646899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de0e5c-cb1d-4138-96c8-175b363b644a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:50.679379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb1ce2-dc6b-42de-a422-5caf96f62e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:50.680037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb1ce2-dc6b-42de-a422-5caf96f62e42 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:50.680319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb1ce2-dc6b-42de-a422-5caf96f62e42 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:50.680607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb1ce2-dc6b-42de-a422-5caf96f62e42 None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:50.681442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb1ce2-dc6b-42de-a422-5caf96f62e42 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:50.681749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb1ce2-dc6b-42de-a422-5caf96f62e42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:50.683161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb1ce2-dc6b-42de-a422-5caf96f62e42 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:50.699870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 536/1092] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Jul 25 08:33:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:50.705638 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14cfb0-7918-48a3-86e8-f0dff7addc00 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:50.736472 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:50.805314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14cfb0-7918-48a3-86e8-f0dff7addc00 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:50.837306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941cb2d-9282-43b4-83dd-76b1891a2af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:50.837947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941cb2d-9282-43b4-83dd-76b1891a2af3 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:50.838234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941cb2d-9282-43b4-83dd-76b1891a2af3 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:50.838515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941cb2d-9282-43b4-83dd-76b1891a2af3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:50.905825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941cb2d-9282-43b4-83dd-76b1891a2af3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:50.907881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941cb2d-9282-43b4-83dd-76b1891a2af3 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:50.938306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:51.008073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 557/1093] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:33:51.013722 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb1ce2-dc6b-42de-a422-5caf96f62e42 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:51.110417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb1ce2-dc6b-42de-a422-5caf96f62e42 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:51.132255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:51.144324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d52ae5-b1c2-4e88-8773-b5fe915d9940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:51.145141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d52ae5-b1c2-4e88-8773-b5fe915d9940 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:51.145427 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d52ae5-b1c2-4e88-8773-b5fe915d9940 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:51.145705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d52ae5-b1c2-4e88-8773-b5fe915d9940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:51.155718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d52ae5-b1c2-4e88-8773-b5fe915d9940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:51.157261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d52ae5-b1c2-4e88-8773-b5fe915d9940 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-f9d52ae5-b1c2-4e88-8773-b5fe915d9940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:33:51.174561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:51.175573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:51.176675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:33:51.176675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:33:51.176675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:33:51.176675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:51.180598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 537/1094] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:51.181341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941cb2d-9282-43b4-83dd-76b1891a2af3 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:51.277538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941cb2d-9282-43b4-83dd-76b1891a2af3 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:51.309509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:51.315692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3d0c6-0e2a-4434-8ba4-c3f96d395e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:51.316391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3d0c6-0e2a-4434-8ba4-c3f96d395e94 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:51.316712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3d0c6-0e2a-4434-8ba4-c3f96d395e94 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:51.316996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3d0c6-0e2a-4434-8ba4-c3f96d395e94 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:51.317755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3d0c6-0e2a-4434-8ba4-c3f96d395e94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:51.318070 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3d0c6-0e2a-4434-8ba4-c3f96d395e94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:51.319455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3d0c6-0e2a-4434-8ba4-c3f96d395e94 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:51.331166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 558/1095] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:51.336599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d52ae5-b1c2-4e88-8773-b5fe915d9940 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:51.432888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d52ae5-b1c2-4e88-8773-b5fe915d9940 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:51.465802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba31b73-adae-4280-830a-e22de2d76e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:51.466588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba31b73-adae-4280-830a-e22de2d76e5c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:51.466871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba31b73-adae-4280-830a-e22de2d76e5c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:51.467146 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba31b73-adae-4280-830a-e22de2d76e5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:51.473749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba31b73-adae-4280-830a-e22de2d76e5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:51.475195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba31b73-adae-4280-830a-e22de2d76e5c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-8ba31b73-adae-4280-830a-e22de2d76e5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:33:51.489344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:51.490350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:51.491219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:33:51.491219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:33:51.491219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:33:51.491219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:33:51.491219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:33:51.491219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:33:51.491219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:33:51.491219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:51.491219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 538/1096] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:33:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:51.496224 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3d0c6-0e2a-4434-8ba4-c3f96d395e94 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:51.593569 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3d0c6-0e2a-4434-8ba4-c3f96d395e94 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:51.626679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee433e-5f78-4c8b-b9f6-4ff167c69399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:51.627340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee433e-5f78-4c8b-b9f6-4ff167c69399 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:51.627630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee433e-5f78-4c8b-b9f6-4ff167c69399 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:51.627909 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee433e-5f78-4c8b-b9f6-4ff167c69399 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:51.628736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee433e-5f78-4c8b-b9f6-4ff167c69399 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:51.629047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee433e-5f78-4c8b-b9f6-4ff167c69399 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:51.630465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee433e-5f78-4c8b-b9f6-4ff167c69399 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:51.642445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 559/1097] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:33:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:51.648263 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba31b73-adae-4280-830a-e22de2d76e5c None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:51.683285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:51.749330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba31b73-adae-4280-830a-e22de2d76e5c None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:51.782573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ff40c-22b4-4dec-858f-bc6b1206c57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:51.783240 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ff40c-22b4-4dec-858f-bc6b1206c57b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:51.783514 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ff40c-22b4-4dec-858f-bc6b1206c57b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:51.783793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ff40c-22b4-4dec-858f-bc6b1206c57b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:51.784654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ff40c-22b4-4dec-858f-bc6b1206c57b None admin] RBAC: Adding query filter params (role.id=1ac8ff72ed404116b52f57f20668457d, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=8b476d6b82ae464b81fc8498dc0e68c7) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:51.784969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ff40c-22b4-4dec-858f-bc6b1206c57b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:51.786214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd9b47-6c28-4ea4-aca6-a08583b67815 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:51.786390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ff40c-22b4-4dec-858f-bc6b1206c57b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:51.786571 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd9b47-6c28-4ea4-aca6-a08583b67815 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:51.786776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd9b47-6c28-4ea4-aca6-a08583b67815 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:51.788154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 560/1098] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:33:51.792728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 539/1099] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:51 2022] GET /identity/v3/role_assignments?user.id=8b476d6b82ae464b81fc8498dc0e68c7&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=1ac8ff72ed404116b52f57f20668457d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:51.792880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba470c1-4da8-4066-a1bc-cd2c18f0814e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:51.793169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba470c1-4da8-4066-a1bc-cd2c18f0814e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:51.793449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba470c1-4da8-4066-a1bc-cd2c18f0814e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:51.812768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-dba470c1-4da8-4066-a1bc-cd2c18f0814e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:51.814003 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-dba470c1-4da8-4066-a1bc-cd2c18f0814e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:51.865409 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:51.886163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 561/1100] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 08:33:51.910767 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ff40c-22b4-4dec-858f-bc6b1206c57b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:52.008179 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:52.013106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ff40c-22b4-4dec-858f-bc6b1206c57b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:52.047881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0e61b-1155-4174-896a-d9b304abcc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:52.048701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0e61b-1155-4174-896a-d9b304abcc68 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:52.048983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0e61b-1155-4174-896a-d9b304abcc68 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:52.049253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0e61b-1155-4174-896a-d9b304abcc68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:52.059602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0e61b-1155-4174-896a-d9b304abcc68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-40d0e61b-1155-4174-896a-d9b304abcc68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:33:52.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:33:52.068553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:33:52.069523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 540/1101] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:33:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:33:52.074785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba470c1-4da8-4066-a1bc-cd2c18f0814e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:52.177419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba470c1-4da8-4066-a1bc-cd2c18f0814e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:52.211533 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e372a13-4ccc-410a-9088-33179e3b3216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:52.212209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e372a13-4ccc-410a-9088-33179e3b3216 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:52.212495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e372a13-4ccc-410a-9088-33179e3b3216 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:52.212802 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e372a13-4ccc-410a-9088-33179e3b3216 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:52.213559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e372a13-4ccc-410a-9088-33179e3b3216 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:52.213867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e372a13-4ccc-410a-9088-33179e3b3216 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:52.215285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e372a13-4ccc-410a-9088-33179e3b3216 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:52.224222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 562/1102] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:33:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:52.230250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0e61b-1155-4174-896a-d9b304abcc68 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:52.338173 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0e61b-1155-4174-896a-d9b304abcc68 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:52.373206 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76243a2-d24a-49eb-9429-7dda3e832536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:52.374071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76243a2-d24a-49eb-9429-7dda3e832536 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:52.374353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76243a2-d24a-49eb-9429-7dda3e832536 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:52.374628 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76243a2-d24a-49eb-9429-7dda3e832536 None admin] PATH_INFO: `/v3/users/36541892a93347d5801f6d5229e9dbab` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:52.376323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76243a2-d24a-49eb-9429-7dda3e832536 None admin] RBAC: Authorizing `identity:get_user(user_id=36541892a93347d5801f6d5229e9dbab)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:52.377780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76243a2-d24a-49eb-9429-7dda3e832536 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:52.379415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 541/1103] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:33:52 2022] GET /identity/v3/users/36541892a93347d5801f6d5229e9dbab => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:52.385354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e372a13-4ccc-410a-9088-33179e3b3216 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:52.433034 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:52.489558 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e372a13-4ccc-410a-9088-33179e3b3216 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:52.524389 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7ff62-7090-4a15-818f-19ce16ddcf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:52.525116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7ff62-7090-4a15-818f-19ce16ddcf3d None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:52.525461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7ff62-7090-4a15-818f-19ce16ddcf3d None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:52.526006 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7ff62-7090-4a15-818f-19ce16ddcf3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:52.594138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f7ff62-7090-4a15-818f-19ce16ddcf3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:52.602119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f7ff62-7090-4a15-818f-19ce16ddcf3d None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:52.646462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:52.700769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 563/1104] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:33:52.706775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76243a2-d24a-49eb-9429-7dda3e832536 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:52.733628 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:52.806575 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76243a2-d24a-49eb-9429-7dda3e832536 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:52.838903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3c032-991e-42f3-97bf-285fe460eeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:52.839748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3c032-991e-42f3-97bf-285fe460eeee None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:52.840031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3c032-991e-42f3-97bf-285fe460eeee None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:52.840311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3c032-991e-42f3-97bf-285fe460eeee None admin] PATH_INFO: `/v3/projects/55d8e4202835461682a7999ca8a66e2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:52.841523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3c032-991e-42f3-97bf-285fe460eeee None admin] RBAC: Authorizing `identity:get_project(project_id=55d8e4202835461682a7999ca8a66e2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:52.842949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3c032-991e-42f3-97bf-285fe460eeee None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:52.844867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 542/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:33:52 2022] GET /identity/v3/projects/55d8e4202835461682a7999ca8a66e2a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:52.850696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7ff62-7090-4a15-818f-19ce16ddcf3d None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:52.946653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7ff62-7090-4a15-818f-19ce16ddcf3d None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:52.979085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04ddf8-0a51-4f66-a4f2-4c29c53d59d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:52.979727 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04ddf8-0a51-4f66-a4f2-4c29c53d59d7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:52.980014 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04ddf8-0a51-4f66-a4f2-4c29c53d59d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:52.980288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04ddf8-0a51-4f66-a4f2-4c29c53d59d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:52.981141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04ddf8-0a51-4f66-a4f2-4c29c53d59d7 None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=55d8e4202835461682a7999ca8a66e2a, user.id=36541892a93347d5801f6d5229e9dbab) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:52.981451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04ddf8-0a51-4f66-a4f2-4c29c53d59d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:52.982854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04ddf8-0a51-4f66-a4f2-4c29c53d59d7 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:52.989055 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 564/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:52 2022] GET /identity/v3/role_assignments?user.id=36541892a93347d5801f6d5229e9dbab&scope.project.id=55d8e4202835461682a7999ca8a66e2a&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:54.972726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed31eeb-58c5-467f-b2f8-ba3587e8536f None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:54.973251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed31eeb-58c5-467f-b2f8-ba3587e8536f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:54.973344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed31eeb-58c5-467f-b2f8-ba3587e8536f None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:54.974651 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 543/1107] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:54.979721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54573c6b-a4c7-4c32-9150-3d91c9d58b27 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:54.980090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54573c6b-a4c7-4c32-9150-3d91c9d58b27 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:54.980282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54573c6b-a4c7-4c32-9150-3d91c9d58b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:54.997914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-54573c6b-a4c7-4c32-9150-3d91c9d58b27 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:54.999099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-54573c6b-a4c7-4c32-9150-3d91c9d58b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:55.001099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:55.065756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 565/1108] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:55.089604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed31eeb-58c5-467f-b2f8-ba3587e8536f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:55.181268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:55.188985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed31eeb-58c5-467f-b2f8-ba3587e8536f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:55.222222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133eba7c-e925-413a-90ec-e7fe5952e2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:55.223042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133eba7c-e925-413a-90ec-e7fe5952e2e2 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:55.223274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133eba7c-e925-413a-90ec-e7fe5952e2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:55.223548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133eba7c-e925-413a-90ec-e7fe5952e2e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:55.224796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133eba7c-e925-413a-90ec-e7fe5952e2e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:55.226215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133eba7c-e925-413a-90ec-e7fe5952e2e2 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:55.227795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 544/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:33:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:55.233400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54573c6b-a4c7-4c32-9150-3d91c9d58b27 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:55.331596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54573c6b-a4c7-4c32-9150-3d91c9d58b27 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:55.364132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94f376-4b2b-4e8d-8a67-491aecbed862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:55.364832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94f376-4b2b-4e8d-8a67-491aecbed862 None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:55.365129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94f376-4b2b-4e8d-8a67-491aecbed862 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:55.365409 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94f376-4b2b-4e8d-8a67-491aecbed862 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:55.366135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94f376-4b2b-4e8d-8a67-491aecbed862 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:55.367567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94f376-4b2b-4e8d-8a67-491aecbed862 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:55.377633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:55.389290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 566/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:33:55 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:33:57.379021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a402eba-7124-46ce-90cd-6c0e821a1381 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:57.379504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a402eba-7124-46ce-90cd-6c0e821a1381 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:57.379704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a402eba-7124-46ce-90cd-6c0e821a1381 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:57.381208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 545/1111] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:33:57.386218 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed07629-fd25-4f37-9152-d1040461262d None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:57.386583 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed07629-fd25-4f37-9152-d1040461262d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:57.386774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed07629-fd25-4f37-9152-d1040461262d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:57.404685 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-2ed07629-fd25-4f37-9152-d1040461262d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:33:57.405857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2ed07629-fd25-4f37-9152-d1040461262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:33:57.470132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 567/1112] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:33:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:33:57.492770 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a402eba-7124-46ce-90cd-6c0e821a1381 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:57.590010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a402eba-7124-46ce-90cd-6c0e821a1381 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:57.622736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8e760-d9f7-423e-bd0b-edfeb79943de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:57.623577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8e760-d9f7-423e-bd0b-edfeb79943de None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:57.623862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8e760-d9f7-423e-bd0b-edfeb79943de None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:57.624139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8e760-d9f7-423e-bd0b-edfeb79943de None admin] PATH_INFO: `/v3/roles/3ce6c4ec0ed142828cb20882e342328e` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:57.625699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8e760-d9f7-423e-bd0b-edfeb79943de None admin] RBAC: Authorizing `identity:get_role(role_id=3ce6c4ec0ed142828cb20882e342328e)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:57.627059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8e760-d9f7-423e-bd0b-edfeb79943de None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:57.628402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 546/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:33:57 2022] GET /identity/v3/roles/3ce6c4ec0ed142828cb20882e342328e => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:57.633954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed07629-fd25-4f37-9152-d1040461262d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:57.728302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed07629-fd25-4f37-9152-d1040461262d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:57.750878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:57.761929 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911143d-58d3-4717-ad99-0beaf83ddb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:57.762784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911143d-58d3-4717-ad99-0beaf83ddb53 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:57.763064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911143d-58d3-4717-ad99-0beaf83ddb53 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:57.763333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911143d-58d3-4717-ad99-0beaf83ddb53 None admin] PATH_INFO: `/v3/users/740caab827dc4e21980d88db06349065` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:57.776130 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911143d-58d3-4717-ad99-0beaf83ddb53 None admin] RBAC: Authorizing `identity:get_user(user_id=740caab827dc4e21980d88db06349065)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:57.777566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911143d-58d3-4717-ad99-0beaf83ddb53 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:57.779182 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 568/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:33:57 2022] GET /identity/v3/users/740caab827dc4e21980d88db06349065 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:57.785352 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8e760-d9f7-423e-bd0b-edfeb79943de None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:57.815377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:57.882744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8e760-d9f7-423e-bd0b-edfeb79943de None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:57.915908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09314769-8fce-4a5b-892a-cb2226652c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:57.916543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09314769-8fce-4a5b-892a-cb2226652c28 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:57.916861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09314769-8fce-4a5b-892a-cb2226652c28 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:57.917137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09314769-8fce-4a5b-892a-cb2226652c28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:57.981213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09314769-8fce-4a5b-892a-cb2226652c28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:57.982634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09314769-8fce-4a5b-892a-cb2226652c28 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:58.009853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:58.079244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 547/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:33:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:33:58.085176 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911143d-58d3-4717-ad99-0beaf83ddb53 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:58.180993 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911143d-58d3-4717-ad99-0beaf83ddb53 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:58.213294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cf281-6607-4982-bb98-0e01ff471491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:58.214123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cf281-6607-4982-bb98-0e01ff471491 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:58.214406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cf281-6607-4982-bb98-0e01ff471491 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:58.214676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cf281-6607-4982-bb98-0e01ff471491 None admin] PATH_INFO: `/v3/projects/55d8e4202835461682a7999ca8a66e2a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:58.215823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cf281-6607-4982-bb98-0e01ff471491 None admin] RBAC: Authorizing `identity:get_project(project_id=55d8e4202835461682a7999ca8a66e2a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:58.217259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cf281-6607-4982-bb98-0e01ff471491 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:58.219536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 569/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:33:58 2022] GET /identity/v3/projects/55d8e4202835461682a7999ca8a66e2a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:33:58.225400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09314769-8fce-4a5b-892a-cb2226652c28 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:33:58.321259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09314769-8fce-4a5b-892a-cb2226652c28 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:33:58.349149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:33:58.355433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53e6a6-287b-4b90-a7cd-21955953e7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:33:58.356086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53e6a6-287b-4b90-a7cd-21955953e7a0 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:33:58.356372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53e6a6-287b-4b90-a7cd-21955953e7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:33:58.356680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53e6a6-287b-4b90-a7cd-21955953e7a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:33:58.357515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53e6a6-287b-4b90-a7cd-21955953e7a0 None admin] RBAC: Adding query filter params (role.id=3ce6c4ec0ed142828cb20882e342328e, scope.project.id=55d8e4202835461682a7999ca8a66e2a, user.id=740caab827dc4e21980d88db06349065) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:33:58.357832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53e6a6-287b-4b90-a7cd-21955953e7a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:33:58.359281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53e6a6-287b-4b90-a7cd-21955953e7a0 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:33:58.365007 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 548/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:33:58 2022] GET /identity/v3/role_assignments?user.id=740caab827dc4e21980d88db06349065&scope.project.id=55d8e4202835461682a7999ca8a66e2a&role.id=3ce6c4ec0ed142828cb20882e342328e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:00.289031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a293a9-4b07-4ed3-b82b-12a52a14ee4e None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:00.289561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a293a9-4b07-4ed3-b82b-12a52a14ee4e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:00.289561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a293a9-4b07-4ed3-b82b-12a52a14ee4e None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:00.290962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 570/1118] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:00.296123 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852da82-e1b1-4087-9d9d-4f4e0fc7893d None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:00.296496 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852da82-e1b1-4087-9d9d-4f4e0fc7893d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:00.296705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852da82-e1b1-4087-9d9d-4f4e0fc7893d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:00.314420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-e852da82-e1b1-4087-9d9d-4f4e0fc7893d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:00.315592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e852da82-e1b1-4087-9d9d-4f4e0fc7893d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:00.379584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 549/1119] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:00.402134 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a293a9-4b07-4ed3-b82b-12a52a14ee4e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:00.451197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:00.500785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a293a9-4b07-4ed3-b82b-12a52a14ee4e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:00.533701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ab7f9-55ec-4300-8975-bcd2fc091060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:00.534533 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ab7f9-55ec-4300-8975-bcd2fc091060 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:00.534813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ab7f9-55ec-4300-8975-bcd2fc091060 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:00.535086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ab7f9-55ec-4300-8975-bcd2fc091060 None admin] PATH_INFO: `/v3/roles/3ce6c4ec0ed142828cb20882e342328e` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:00.536531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ab7f9-55ec-4300-8975-bcd2fc091060 None admin] RBAC: Authorizing `identity:get_role(role_id=3ce6c4ec0ed142828cb20882e342328e)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:00.537907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ab7f9-55ec-4300-8975-bcd2fc091060 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:00.539309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 571/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:34:00 2022] GET /identity/v3/roles/3ce6c4ec0ed142828cb20882e342328e => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:00.545071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852da82-e1b1-4087-9d9d-4f4e0fc7893d None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:00.638935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852da82-e1b1-4087-9d9d-4f4e0fc7893d None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:00.661674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:00.675776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ae528-c302-4e67-8ba5-69c8e7bc383d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:00.676673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ae528-c302-4e67-8ba5-69c8e7bc383d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:00.676957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ae528-c302-4e67-8ba5-69c8e7bc383d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:00.677232 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ae528-c302-4e67-8ba5-69c8e7bc383d None admin] PATH_INFO: `/v3/users/740caab827dc4e21980d88db06349065` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:00.678408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ae528-c302-4e67-8ba5-69c8e7bc383d None admin] RBAC: Authorizing `identity:get_user(user_id=740caab827dc4e21980d88db06349065)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:00.679823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ae528-c302-4e67-8ba5-69c8e7bc383d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:00.681467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 550/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:34:00 2022] GET /identity/v3/users/740caab827dc4e21980d88db06349065 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:00.687726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ab7f9-55ec-4300-8975-bcd2fc091060 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:00.782631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ab7f9-55ec-4300-8975-bcd2fc091060 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:00.798946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:00.815614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b193f8-ae28-4626-9556-66c1eaa4ff2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:00.816456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b193f8-ae28-4626-9556-66c1eaa4ff2e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:00.816775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b193f8-ae28-4626-9556-66c1eaa4ff2e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:00.817050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b193f8-ae28-4626-9556-66c1eaa4ff2e None admin] PATH_INFO: `/v3/projects/55d8e4202835461682a7999ca8a66e2a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:00.818208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b193f8-ae28-4626-9556-66c1eaa4ff2e None admin] RBAC: Authorizing `identity:get_project(project_id=55d8e4202835461682a7999ca8a66e2a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:00.819619 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b193f8-ae28-4626-9556-66c1eaa4ff2e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:00.821337 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 572/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:34:00 2022] GET /identity/v3/projects/55d8e4202835461682a7999ca8a66e2a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:00.827266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ae528-c302-4e67-8ba5-69c8e7bc383d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:00.923128 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ae528-c302-4e67-8ba5-69c8e7bc383d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:00.956346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3c10b-c193-4467-bd87-89df6d0d0c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:00.957254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3c10b-c193-4467-bd87-89df6d0d0c4f None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:00.957546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3c10b-c193-4467-bd87-89df6d0d0c4f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:00.957825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3c10b-c193-4467-bd87-89df6d0d0c4f None admin] PATH_INFO: `/v3/projects/55d8e4202835461682a7999ca8a66e2a/users/740caab827dc4e21980d88db06349065/roles/3ce6c4ec0ed142828cb20882e342328e` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:00.960008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3c10b-c193-4467-bd87-89df6d0d0c4f None admin] RBAC: Authorizing `identity:create_grant(project_id=55d8e4202835461682a7999ca8a66e2a, user_id=740caab827dc4e21980d88db06349065, role_id=3ce6c4ec0ed142828cb20882e342328e)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:00.961448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3c10b-c193-4467-bd87-89df6d0d0c4f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:00.973684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 551/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:34:00 2022] PUT /identity/v3/projects/55d8e4202835461682a7999ca8a66e2a/users/740caab827dc4e21980d88db06349065/roles/3ce6c4ec0ed142828cb20882e342328e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:34:02.905098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabc07e-bd61-487b-bd78-63e59c03378e None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:02.905644 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabc07e-bd61-487b-bd78-63e59c03378e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:02.905644 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabc07e-bd61-487b-bd78-63e59c03378e None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:02.907516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 573/1124] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:02.912532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3a946-c9d1-4649-8edc-8e3f80d33d44 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:02.912993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3a946-c9d1-4649-8edc-8e3f80d33d44 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:02.913117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3a946-c9d1-4649-8edc-8e3f80d33d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:02.931355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-b7e3a946-c9d1-4649-8edc-8e3f80d33d44 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:02.932532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b7e3a946-c9d1-4649-8edc-8e3f80d33d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:02.977188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:02.998048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 552/1125] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:03.019953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabc07e-bd61-487b-bd78-63e59c03378e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:03.117761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabc07e-bd61-487b-bd78-63e59c03378e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:03.150042 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13252c46-701f-490a-a641-6177c7a449b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:03.151132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13252c46-701f-490a-a641-6177c7a449b5 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:03.151413 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13252c46-701f-490a-a641-6177c7a449b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:03.151686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13252c46-701f-490a-a641-6177c7a449b5 None admin] PATH_INFO: `/v3/roles/3ce6c4ec0ed142828cb20882e342328e` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:03.153176 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13252c46-701f-490a-a641-6177c7a449b5 None admin] RBAC: Authorizing `identity:get_role(role_id=3ce6c4ec0ed142828cb20882e342328e)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:03.154531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13252c46-701f-490a-a641-6177c7a449b5 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:03.156439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 574/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:34:03 2022] GET /identity/v3/roles/3ce6c4ec0ed142828cb20882e342328e => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:03.161979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3a946-c9d1-4649-8edc-8e3f80d33d44 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:03.256064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3a946-c9d1-4649-8edc-8e3f80d33d44 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:03.289821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545a316-2e53-4ac2-a14b-1d41cb27cda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:03.290697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545a316-2e53-4ac2-a14b-1d41cb27cda2 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:03.290986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545a316-2e53-4ac2-a14b-1d41cb27cda2 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:03.291267 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545a316-2e53-4ac2-a14b-1d41cb27cda2 None admin] PATH_INFO: `/v3/users/740caab827dc4e21980d88db06349065` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:03.292467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545a316-2e53-4ac2-a14b-1d41cb27cda2 None admin] RBAC: Authorizing `identity:get_user(user_id=740caab827dc4e21980d88db06349065)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:03.293932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545a316-2e53-4ac2-a14b-1d41cb27cda2 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:03.295576 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 553/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:34:03 2022] GET /identity/v3/users/740caab827dc4e21980d88db06349065 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:03.301151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13252c46-701f-490a-a641-6177c7a449b5 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:03.357281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:03.401967 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13252c46-701f-490a-a641-6177c7a449b5 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:03.434591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0698d-225b-4e83-b793-792d1349de2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:03.435183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0698d-225b-4e83-b793-792d1349de2b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:03.435469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0698d-225b-4e83-b793-792d1349de2b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:03.435748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0698d-225b-4e83-b793-792d1349de2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:03.500882 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0698d-225b-4e83-b793-792d1349de2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:03.502345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0698d-225b-4e83-b793-792d1349de2b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:03.553599 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:03.597874 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 575/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:34:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:34:03.603741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545a316-2e53-4ac2-a14b-1d41cb27cda2 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:03.652545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:03.699607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545a316-2e53-4ac2-a14b-1d41cb27cda2 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:03.732076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe805e-64d0-4483-b021-bdab23b50e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:03.732952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe805e-64d0-4483-b021-bdab23b50e89 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:03.733238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe805e-64d0-4483-b021-bdab23b50e89 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:03.733515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe805e-64d0-4483-b021-bdab23b50e89 None admin] PATH_INFO: `/v3/projects/55d8e4202835461682a7999ca8a66e2a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:03.734716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe805e-64d0-4483-b021-bdab23b50e89 None admin] RBAC: Authorizing `identity:get_project(project_id=55d8e4202835461682a7999ca8a66e2a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:03.736131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe805e-64d0-4483-b021-bdab23b50e89 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:03.737826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 554/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:34:03 2022] GET /identity/v3/projects/55d8e4202835461682a7999ca8a66e2a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:03.743808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0698d-225b-4e83-b793-792d1349de2b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:03.840489 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0698d-225b-4e83-b793-792d1349de2b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:03.872877 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54954f46-4561-4818-891c-64f1f5121d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:03.873519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54954f46-4561-4818-891c-64f1f5121d0e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:03.873805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54954f46-4561-4818-891c-64f1f5121d0e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:03.874086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54954f46-4561-4818-891c-64f1f5121d0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:03.875450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54954f46-4561-4818-891c-64f1f5121d0e None admin] RBAC: Adding query filter params (role.id=3ce6c4ec0ed142828cb20882e342328e, scope.project.id=55d8e4202835461682a7999ca8a66e2a, user.id=740caab827dc4e21980d88db06349065) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:34:03.875757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54954f46-4561-4818-891c-64f1f5121d0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:03.877206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54954f46-4561-4818-891c-64f1f5121d0e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:03.882887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 576/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:34:03 2022] GET /identity/v3/role_assignments?user.id=740caab827dc4e21980d88db06349065&scope.project.id=55d8e4202835461682a7999ca8a66e2a&role.id=3ce6c4ec0ed142828cb20882e342328e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:05.873260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e97a8-e832-46a7-a237-a6030f0dda18 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:05.873789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e97a8-e832-46a7-a237-a6030f0dda18 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:05.873901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e97a8-e832-46a7-a237-a6030f0dda18 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:05.875202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 555/1131] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:05.880285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6897df-6de5-4497-bf8e-9593e00db096 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:05.880721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6897df-6de5-4497-bf8e-9593e00db096 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:05.880890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6897df-6de5-4497-bf8e-9593e00db096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:05.898549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-8a6897df-6de5-4497-bf8e-9593e00db096 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:05.899741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8a6897df-6de5-4497-bf8e-9593e00db096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:05.913370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:05.965511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 577/1132] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:05.989190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e97a8-e832-46a7-a237-a6030f0dda18 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:06.087395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e97a8-e832-46a7-a237-a6030f0dda18 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:06.104227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:06.121339 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6997348-86c2-4d75-822f-6a51b08af920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:06.122114 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6997348-86c2-4d75-822f-6a51b08af920 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:06.122398 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6997348-86c2-4d75-822f-6a51b08af920 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:06.122673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6997348-86c2-4d75-822f-6a51b08af920 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:06.123893 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6997348-86c2-4d75-822f-6a51b08af920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:06.125313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6997348-86c2-4d75-822f-6a51b08af920 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:06.126904 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 556/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:34:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:06.132638 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6897df-6de5-4497-bf8e-9593e00db096 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:06.230177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6897df-6de5-4497-bf8e-9593e00db096 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:06.262566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be001fd-61b7-45a0-97ec-7fc1bdb185ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:06.263221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be001fd-61b7-45a0-97ec-7fc1bdb185ee None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:06.263508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be001fd-61b7-45a0-97ec-7fc1bdb185ee None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:06.263787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be001fd-61b7-45a0-97ec-7fc1bdb185ee None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:06.264496 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be001fd-61b7-45a0-97ec-7fc1bdb185ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:06.266452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be001fd-61b7-45a0-97ec-7fc1bdb185ee None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:06.287530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:06.294937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 578/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:34:06 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:34:08.293532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9f0b0-c4b5-427c-a2b7-31f292424f99 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:08.294070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9f0b0-c4b5-427c-a2b7-31f292424f99 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:08.294070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9f0b0-c4b5-427c-a2b7-31f292424f99 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:08.295457 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 557/1135] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:08.301612 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1425b57-988d-4fff-8b89-50da96551a23 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:08.302043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1425b57-988d-4fff-8b89-50da96551a23 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:08.302179 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1425b57-988d-4fff-8b89-50da96551a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:08.320504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-b1425b57-988d-4fff-8b89-50da96551a23 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:08.321724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b1425b57-988d-4fff-8b89-50da96551a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:08.387212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 579/1136] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:08.411550 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9f0b0-c4b5-427c-a2b7-31f292424f99 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:08.508931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9f0b0-c4b5-427c-a2b7-31f292424f99 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:08.541053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0bdd0-dc88-48f8-bbe4-13335d2b2a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:08.541828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0bdd0-dc88-48f8-bbe4-13335d2b2a5e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:08.542107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0bdd0-dc88-48f8-bbe4-13335d2b2a5e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:08.542380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0bdd0-dc88-48f8-bbe4-13335d2b2a5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:08.543589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0bdd0-dc88-48f8-bbe4-13335d2b2a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:08.544980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0bdd0-dc88-48f8-bbe4-13335d2b2a5e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:08.547278 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 558/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:34:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:08.552515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1425b57-988d-4fff-8b89-50da96551a23 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:08.647891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1425b57-988d-4fff-8b89-50da96551a23 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:08.675397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:08.681519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cd6ac-5af2-42da-886a-ac2b9c0ffcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:08.682150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cd6ac-5af2-42da-886a-ac2b9c0ffcfe None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:08.682432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cd6ac-5af2-42da-886a-ac2b9c0ffcfe None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:08.682706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cd6ac-5af2-42da-886a-ac2b9c0ffcfe None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:08.683405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cd6ac-5af2-42da-886a-ac2b9c0ffcfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:08.684850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cd6ac-5af2-42da-886a-ac2b9c0ffcfe None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:08.705825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 580/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:34:08 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:34:10.702983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d705624-6c96-45ef-8f2c-a41640a16028 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:10.703533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d705624-6c96-45ef-8f2c-a41640a16028 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:10.703533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d705624-6c96-45ef-8f2c-a41640a16028 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:10.704963 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 559/1139] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:10.709988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128662a-57b4-4e5c-ae40-810cc1030197 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:10.710421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128662a-57b4-4e5c-ae40-810cc1030197 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:10.710554 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128662a-57b4-4e5c-ae40-810cc1030197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:10.728944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-b128662a-57b4-4e5c-ae40-810cc1030197 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:10.730169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b128662a-57b4-4e5c-ae40-810cc1030197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:10.794691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 581/1140] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:10.817004 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d705624-6c96-45ef-8f2c-a41640a16028 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:10.871045 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:10.916185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d705624-6c96-45ef-8f2c-a41640a16028 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:10.948472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90220f04-6a14-4249-933a-94a1bc44658b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:10.949269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90220f04-6a14-4249-933a-94a1bc44658b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:10.949550 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90220f04-6a14-4249-933a-94a1bc44658b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:10.949822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90220f04-6a14-4249-933a-94a1bc44658b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:10.960076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90220f04-6a14-4249-933a-94a1bc44658b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-90220f04-6a14-4249-933a-94a1bc44658b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:34:10.967564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:34:10.969408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:34:10.970710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 560/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:34:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:34:10.975289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128662a-57b4-4e5c-ae40-810cc1030197 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:11.071624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128662a-57b4-4e5c-ae40-810cc1030197 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:11.087777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:11.106160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c719a-81f6-476d-9991-973271f0b2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:11.106821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c719a-81f6-476d-9991-973271f0b2b5 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:11.107102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c719a-81f6-476d-9991-973271f0b2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:11.107374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c719a-81f6-476d-9991-973271f0b2b5 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:11.108108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c719a-81f6-476d-9991-973271f0b2b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:34:11.108409 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c719a-81f6-476d-9991-973271f0b2b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:11.109829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c719a-81f6-476d-9991-973271f0b2b5 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:11.117721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 582/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:34:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:11.123202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90220f04-6a14-4249-933a-94a1bc44658b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:11.219615 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90220f04-6a14-4249-933a-94a1bc44658b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:11.232413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:11.252706 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326e924-a656-48d8-a10c-8128c28677fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:11.253551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326e924-a656-48d8-a10c-8128c28677fb None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:11.253830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326e924-a656-48d8-a10c-8128c28677fb None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:11.254101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326e924-a656-48d8-a10c-8128c28677fb None admin] PATH_INFO: `/v3/users/161c333f8a61497f8d62b8d918624a3b` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:11.267221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326e924-a656-48d8-a10c-8128c28677fb None admin] RBAC: Authorizing `identity:get_user(user_id=161c333f8a61497f8d62b8d918624a3b)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:11.268658 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326e924-a656-48d8-a10c-8128c28677fb None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:11.270282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 561/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:34:11 2022] GET /identity/v3/users/161c333f8a61497f8d62b8d918624a3b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:11.276121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c719a-81f6-476d-9991-973271f0b2b5 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:11.373045 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c719a-81f6-476d-9991-973271f0b2b5 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:11.405977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277720a-22e9-4ec4-b99d-9c483128cc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:11.406611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277720a-22e9-4ec4-b99d-9c483128cc28 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:11.406893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277720a-22e9-4ec4-b99d-9c483128cc28 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:11.407166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277720a-22e9-4ec4-b99d-9c483128cc28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:11.450951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:11.472798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277720a-22e9-4ec4-b99d-9c483128cc28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:11.474190 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277720a-22e9-4ec4-b99d-9c483128cc28 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:11.569459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 583/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:34:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:34:11.575275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326e924-a656-48d8-a10c-8128c28677fb None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:11.669789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326e924-a656-48d8-a10c-8128c28677fb None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:11.702535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7b009-42fa-4bc2-beb9-c91ac0b0798c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:11.703367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7b009-42fa-4bc2-beb9-c91ac0b0798c None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:11.703895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7b009-42fa-4bc2-beb9-c91ac0b0798c None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:11.703895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7b009-42fa-4bc2-beb9-c91ac0b0798c None admin] PATH_INFO: `/v3/projects/15920a7756554db9aba27f73384607b9` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:11.705063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7b009-42fa-4bc2-beb9-c91ac0b0798c None admin] RBAC: Authorizing `identity:get_project(project_id=15920a7756554db9aba27f73384607b9)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:11.706473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7b009-42fa-4bc2-beb9-c91ac0b0798c None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:11.708118 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 562/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:34:11 2022] GET /identity/v3/projects/15920a7756554db9aba27f73384607b9 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:11.713994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277720a-22e9-4ec4-b99d-9c483128cc28 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:11.790447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:11.811954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277720a-22e9-4ec4-b99d-9c483128cc28 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:11.844824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad59b0b-4f73-4a4f-b399-47f8b315defb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:11.845470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad59b0b-4f73-4a4f-b399-47f8b315defb None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:11.845756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad59b0b-4f73-4a4f-b399-47f8b315defb None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:11.846039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad59b0b-4f73-4a4f-b399-47f8b315defb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:11.846884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad59b0b-4f73-4a4f-b399-47f8b315defb None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=15920a7756554db9aba27f73384607b9, user.id=161c333f8a61497f8d62b8d918624a3b) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:34:11.847202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad59b0b-4f73-4a4f-b399-47f8b315defb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:11.848641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad59b0b-4f73-4a4f-b399-47f8b315defb None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:11.853959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 584/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:34:11 2022] GET /identity/v3/role_assignments?user.id=161c333f8a61497f8d62b8d918624a3b&scope.project.id=15920a7756554db9aba27f73384607b9&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:13.833594 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ddd2c-d590-429b-84bf-43b85d53c078 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:13.834142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ddd2c-d590-429b-84bf-43b85d53c078 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:13.834142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ddd2c-d590-429b-84bf-43b85d53c078 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:13.835512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 563/1147] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:13.840735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36889600-c52d-4903-85cd-24363511f899 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:13.841120 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36889600-c52d-4903-85cd-24363511f899 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:13.841303 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36889600-c52d-4903-85cd-24363511f899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:13.859524 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-36889600-c52d-4903-85cd-24363511f899 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:13.860728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-36889600-c52d-4903-85cd-24363511f899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:13.924336 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 585/1148] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:13.946109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ddd2c-d590-429b-84bf-43b85d53c078 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:13.987420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:14.043589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ddd2c-d590-429b-84bf-43b85d53c078 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:14.075742 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7f8a7-c079-44ae-80d6-f2eb59237a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:14.076514 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7f8a7-c079-44ae-80d6-f2eb59237a6e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:14.076820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7f8a7-c079-44ae-80d6-f2eb59237a6e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:14.077116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7f8a7-c079-44ae-80d6-f2eb59237a6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:14.086865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7f8a7-c079-44ae-80d6-f2eb59237a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-4be7f8a7-c079-44ae-80d6-f2eb59237a6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:34:14.094926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:34:14.096029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:34:14.097333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 564/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:34:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:34:14.101931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36889600-c52d-4903-85cd-24363511f899 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:14.169029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:14.198085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36889600-c52d-4903-85cd-24363511f899 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:14.230849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f0a4e-77b8-4fa1-8150-a3f289ad51b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:14.231498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f0a4e-77b8-4fa1-8150-a3f289ad51b7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:14.231780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f0a4e-77b8-4fa1-8150-a3f289ad51b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:14.232052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f0a4e-77b8-4fa1-8150-a3f289ad51b7 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:14.232823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f0a4e-77b8-4fa1-8150-a3f289ad51b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:34:14.233128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f0a4e-77b8-4fa1-8150-a3f289ad51b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:14.235082 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f0a4e-77b8-4fa1-8150-a3f289ad51b7 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:14.242949 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 586/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:34:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:34:14.248562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7f8a7-c079-44ae-80d6-f2eb59237a6e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:14.341713 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:14.345426 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7f8a7-c079-44ae-80d6-f2eb59237a6e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:14.378130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53693156-4b39-435e-b1b4-b51c3590d7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:14.378992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53693156-4b39-435e-b1b4-b51c3590d7fa None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:14.379273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53693156-4b39-435e-b1b4-b51c3590d7fa None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:14.379546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53693156-4b39-435e-b1b4-b51c3590d7fa None admin] PATH_INFO: `/v3/users/161c333f8a61497f8d62b8d918624a3b` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:14.380747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53693156-4b39-435e-b1b4-b51c3590d7fa None admin] RBAC: Authorizing `identity:get_user(user_id=161c333f8a61497f8d62b8d918624a3b)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:14.382165 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53693156-4b39-435e-b1b4-b51c3590d7fa None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:14.383771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 565/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:34:14 2022] GET /identity/v3/users/161c333f8a61497f8d62b8d918624a3b => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:14.390019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f0a4e-77b8-4fa1-8150-a3f289ad51b7 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:14.485295 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f0a4e-77b8-4fa1-8150-a3f289ad51b7 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:14.517217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df224fb5-75c5-4652-a283-534156a5cdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:14.518059 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df224fb5-75c5-4652-a283-534156a5cdc9 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:14.518339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df224fb5-75c5-4652-a283-534156a5cdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:14.518614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df224fb5-75c5-4652-a283-534156a5cdc9 None admin] PATH_INFO: `/v3/projects/15920a7756554db9aba27f73384607b9` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:14.519769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df224fb5-75c5-4652-a283-534156a5cdc9 None admin] RBAC: Authorizing `identity:get_project(project_id=15920a7756554db9aba27f73384607b9)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:14.521211 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df224fb5-75c5-4652-a283-534156a5cdc9 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:14.522872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 587/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:34:14 2022] GET /identity/v3/projects/15920a7756554db9aba27f73384607b9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:14.528662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53693156-4b39-435e-b1b4-b51c3590d7fa None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:14.623270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53693156-4b39-435e-b1b4-b51c3590d7fa None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:14.655898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94ca1c-3528-420e-99de-9a9624b4881a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:14.656786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94ca1c-3528-420e-99de-9a9624b4881a None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:14.657066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94ca1c-3528-420e-99de-9a9624b4881a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:14.657335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94ca1c-3528-420e-99de-9a9624b4881a None admin] PATH_INFO: `/v3/projects/15920a7756554db9aba27f73384607b9/users/161c333f8a61497f8d62b8d918624a3b/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:14.659253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94ca1c-3528-420e-99de-9a9624b4881a None admin] RBAC: Authorizing `identity:create_grant(project_id=15920a7756554db9aba27f73384607b9, user_id=161c333f8a61497f8d62b8d918624a3b, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:14.660677 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94ca1c-3528-420e-99de-9a9624b4881a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:14.673777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 566/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:34:14 2022] PUT /identity/v3/projects/15920a7756554db9aba27f73384607b9/users/161c333f8a61497f8d62b8d918624a3b/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:34:16.611302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac2cea-743d-4fab-9349-4a3d6961a28a None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:16.611836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac2cea-743d-4fab-9349-4a3d6961a28a None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:16.611836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac2cea-743d-4fab-9349-4a3d6961a28a None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:16.613210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 588/1154] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:16.618236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb344899-7f79-4b69-b02e-d628aa3964fc None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:16.618664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb344899-7f79-4b69-b02e-d628aa3964fc None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:16.618798 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb344899-7f79-4b69-b02e-d628aa3964fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:16.637483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-cb344899-7f79-4b69-b02e-d628aa3964fc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:16.638656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-cb344899-7f79-4b69-b02e-d628aa3964fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:16.661367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:16.705235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 567/1155] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:16.728128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac2cea-743d-4fab-9349-4a3d6961a28a None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:16.731082 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:16.827875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac2cea-743d-4fab-9349-4a3d6961a28a None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:16.860376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe1012-4930-43e5-9aad-956e776ab083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:16.861181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe1012-4930-43e5-9aad-956e776ab083 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:16.861516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe1012-4930-43e5-9aad-956e776ab083 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:16.861743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe1012-4930-43e5-9aad-956e776ab083 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:16.871452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe1012-4930-43e5-9aad-956e776ab083 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-5ffe1012-4930-43e5-9aad-956e776ab083 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:34:16.879603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:34:16.880716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:34:16.881855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 589/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:34:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:34:16.886652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb344899-7f79-4b69-b02e-d628aa3964fc None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:16.982938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb344899-7f79-4b69-b02e-d628aa3964fc None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:17.016065 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e285f87-46fc-469f-9d2a-71c9ae7ae78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:17.016755 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e285f87-46fc-469f-9d2a-71c9ae7ae78f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:17.017041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e285f87-46fc-469f-9d2a-71c9ae7ae78f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:17.017316 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e285f87-46fc-469f-9d2a-71c9ae7ae78f None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:17.018059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e285f87-46fc-469f-9d2a-71c9ae7ae78f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:34:17.018365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e285f87-46fc-469f-9d2a-71c9ae7ae78f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:17.019722 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e285f87-46fc-469f-9d2a-71c9ae7ae78f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:17.027760 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 568/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:34:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:17.033753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe1012-4930-43e5-9aad-956e776ab083 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:17.095943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:17.133756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe1012-4930-43e5-9aad-956e776ab083 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:17.166913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959e59c-137c-41db-8390-f805a07ee040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:17.167772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959e59c-137c-41db-8390-f805a07ee040 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:17.168057 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959e59c-137c-41db-8390-f805a07ee040 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:17.168333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959e59c-137c-41db-8390-f805a07ee040 None admin] PATH_INFO: `/v3/users/161c333f8a61497f8d62b8d918624a3b` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:17.169541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959e59c-137c-41db-8390-f805a07ee040 None admin] RBAC: Authorizing `identity:get_user(user_id=161c333f8a61497f8d62b8d918624a3b)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:17.171552 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959e59c-137c-41db-8390-f805a07ee040 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:17.173215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 590/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:34:17 2022] GET /identity/v3/users/161c333f8a61497f8d62b8d918624a3b => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:17.178849 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e285f87-46fc-469f-9d2a-71c9ae7ae78f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:17.200214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:17.276151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e285f87-46fc-469f-9d2a-71c9ae7ae78f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:17.309218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319959cf-0bfc-4c81-8673-9b42f08b0b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:17.309871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319959cf-0bfc-4c81-8673-9b42f08b0b16 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:17.310158 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319959cf-0bfc-4c81-8673-9b42f08b0b16 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:17.310438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319959cf-0bfc-4c81-8673-9b42f08b0b16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:17.375612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319959cf-0bfc-4c81-8673-9b42f08b0b16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:17.377091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319959cf-0bfc-4c81-8673-9b42f08b0b16 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:17.397129 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:17.472780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 569/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:34:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:34:17.478590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959e59c-137c-41db-8390-f805a07ee040 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:17.572505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959e59c-137c-41db-8390-f805a07ee040 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:17.600425 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:17.606376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5765a-cab5-4848-8f6c-08d4906e0cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:17.607218 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5765a-cab5-4848-8f6c-08d4906e0cd6 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:17.607497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5765a-cab5-4848-8f6c-08d4906e0cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:17.607768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5765a-cab5-4848-8f6c-08d4906e0cd6 None admin] PATH_INFO: `/v3/projects/15920a7756554db9aba27f73384607b9` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:17.608989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5765a-cab5-4848-8f6c-08d4906e0cd6 None admin] RBAC: Authorizing `identity:get_project(project_id=15920a7756554db9aba27f73384607b9)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:17.610391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5765a-cab5-4848-8f6c-08d4906e0cd6 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:17.612053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 591/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:34:17 2022] GET /identity/v3/projects/15920a7756554db9aba27f73384607b9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:17.618176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319959cf-0bfc-4c81-8673-9b42f08b0b16 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:17.713636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319959cf-0bfc-4c81-8673-9b42f08b0b16 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:17.733506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:17.747193 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e3e43-9fe8-4963-bcbe-14b1f594870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:17.747834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e3e43-9fe8-4963-bcbe-14b1f594870f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:17.748117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e3e43-9fe8-4963-bcbe-14b1f594870f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:17.748392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e3e43-9fe8-4963-bcbe-14b1f594870f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:17.749750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e3e43-9fe8-4963-bcbe-14b1f594870f None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=15920a7756554db9aba27f73384607b9, user.id=161c333f8a61497f8d62b8d918624a3b) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:34:17.750056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e3e43-9fe8-4963-bcbe-14b1f594870f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:17.751497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e3e43-9fe8-4963-bcbe-14b1f594870f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:17.757198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 570/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:34:17 2022] GET /identity/v3/role_assignments?user.id=161c333f8a61497f8d62b8d918624a3b&scope.project.id=15920a7756554db9aba27f73384607b9&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:19.731335 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b692e-e2f8-4303-b441-2cfd3501a224 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:19.731848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b692e-e2f8-4303-b441-2cfd3501a224 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:19.731952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b692e-e2f8-4303-b441-2cfd3501a224 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:19.733853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 592/1162] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:19.739009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f9ff-ccdf-484d-baa7-767d6360c09f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:19.739434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f9ff-ccdf-484d-baa7-767d6360c09f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:19.739568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f9ff-ccdf-484d-baa7-767d6360c09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:19.757799 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-6f64f9ff-ccdf-484d-baa7-767d6360c09f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:19.758963 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-6f64f9ff-ccdf-484d-baa7-767d6360c09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:19.823657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 571/1163] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:19.844858 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b692e-e2f8-4303-b441-2cfd3501a224 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:19.942180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b692e-e2f8-4303-b441-2cfd3501a224 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:19.974086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df9df8-94ad-42e6-b7e5-e5bd07701a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:19.974715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df9df8-94ad-42e6-b7e5-e5bd07701a55 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:19.974998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df9df8-94ad-42e6-b7e5-e5bd07701a55 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:19.975270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df9df8-94ad-42e6-b7e5-e5bd07701a55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:20.037238 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:20.041135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df9df8-94ad-42e6-b7e5-e5bd07701a55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:20.042832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df9df8-94ad-42e6-b7e5-e5bd07701a55 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:20.136491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 593/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:34:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 08:34:20.142358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64f9ff-ccdf-484d-baa7-767d6360c09f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:20.238089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64f9ff-ccdf-484d-baa7-767d6360c09f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:20.251331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:20.270806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc742d-22bc-4d4b-ba8b-dd768cca3930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:20.271598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc742d-22bc-4d4b-ba8b-dd768cca3930 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:20.271881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc742d-22bc-4d4b-ba8b-dd768cca3930 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:20.272156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc742d-22bc-4d4b-ba8b-dd768cca3930 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:20.278958 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc742d-22bc-4d4b-ba8b-dd768cca3930 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:20.280941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc742d-22bc-4d4b-ba8b-dd768cca3930 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-54bc742d-22bc-4d4b-ba8b-dd768cca3930 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:34:20.294830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:34:20.296670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:34:20.297797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 572/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Jul 25 08:34:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:34:20.302573 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df9df8-94ad-42e6-b7e5-e5bd07701a55 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:20.393975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:20.397688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df9df8-94ad-42e6-b7e5-e5bd07701a55 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:20.430641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa867b-bc66-457f-96f5-dd4ce98f8b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:20.431288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa867b-bc66-457f-96f5-dd4ce98f8b63 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:20.431569 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa867b-bc66-457f-96f5-dd4ce98f8b63 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:20.431844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa867b-bc66-457f-96f5-dd4ce98f8b63 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:20.432555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa867b-bc66-457f-96f5-dd4ce98f8b63 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:34:20.432880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa867b-bc66-457f-96f5-dd4ce98f8b63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:20.434226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa867b-bc66-457f-96f5-dd4ce98f8b63 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:20.441159 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 594/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Jul 25 08:34:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:22.385170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c9ba1-bce9-4cc9-baa9-29265cd4860c None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:22.385624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c9ba1-bce9-4cc9-baa9-29265cd4860c None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:22.385732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c9ba1-bce9-4cc9-baa9-29265cd4860c None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:22.387106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 573/1167] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:22.392314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991328f-e78b-4c8d-a847-7ea8d30e056c None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:22.392757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991328f-e78b-4c8d-a847-7ea8d30e056c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:22.392966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991328f-e78b-4c8d-a847-7ea8d30e056c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:22.411280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c991328f-e78b-4c8d-a847-7ea8d30e056c None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:22.412450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c991328f-e78b-4c8d-a847-7ea8d30e056c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:22.476767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 595/1168] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:22.499269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c9ba1-bce9-4cc9-baa9-29265cd4860c None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:22.597778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c9ba1-bce9-4cc9-baa9-29265cd4860c None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:22.630442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89b8a6-2fcb-4ca2-afff-b47166a11329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:22.631098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89b8a6-2fcb-4ca2-afff-b47166a11329 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:22.631385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89b8a6-2fcb-4ca2-afff-b47166a11329 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:22.631664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89b8a6-2fcb-4ca2-afff-b47166a11329 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:22.632402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89b8a6-2fcb-4ca2-afff-b47166a11329 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:22.633789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89b8a6-2fcb-4ca2-afff-b47166a11329 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:22.649902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 574/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Mon Jul 25 08:34:22 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:34:24.629242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1bd1b-028e-4866-b5e2-3899022d174e None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:24.629755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1bd1b-028e-4866-b5e2-3899022d174e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:24.629755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1bd1b-028e-4866-b5e2-3899022d174e None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:24.631091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 596/1170] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:24.636157 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bddda-9d92-422e-b83f-5ef5bc9c8695 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:24.636528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bddda-9d92-422e-b83f-5ef5bc9c8695 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:24.636758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bddda-9d92-422e-b83f-5ef5bc9c8695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:24.655481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-328bddda-9d92-422e-b83f-5ef5bc9c8695 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:24.656692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-328bddda-9d92-422e-b83f-5ef5bc9c8695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:24.682111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:24.722060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 575/1171] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:24.744431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1bd1b-028e-4866-b5e2-3899022d174e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:24.834440 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:24.844020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1bd1b-028e-4866-b5e2-3899022d174e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:24.876207 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370bcaa-9cba-4eb5-b769-419303731fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:24.877079 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370bcaa-9cba-4eb5-b769-419303731fa9 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:24.877358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370bcaa-9cba-4eb5-b769-419303731fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:24.877632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370bcaa-9cba-4eb5-b769-419303731fa9 None admin] PATH_INFO: `/v3/domains/25c11cfccf2145829cfb233e876d4368` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:24.887613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370bcaa-9cba-4eb5-b769-419303731fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=25c11cfccf2145829cfb233e876d4368)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:24.889698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370bcaa-9cba-4eb5-b769-419303731fa9 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:24.891333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 597/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Mon Jul 25 08:34:24 2022] GET /identity/v3/domains/25c11cfccf2145829cfb233e876d4368 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:24.896997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bddda-9d92-422e-b83f-5ef5bc9c8695 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:24.991059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bddda-9d92-422e-b83f-5ef5bc9c8695 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:25.024710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479bd23-395f-422b-a65b-19a065cd65d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:25.025271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479bd23-395f-422b-a65b-19a065cd65d8 None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:25.025551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479bd23-395f-422b-a65b-19a065cd65d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:25.025822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479bd23-395f-422b-a65b-19a065cd65d8 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:25.026526 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479bd23-395f-422b-a65b-19a065cd65d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:25.027923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479bd23-395f-422b-a65b-19a065cd65d8 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:25.053066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 576/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:34:24 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:34:27.041954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc34373-8188-4a81-8eb2-7ece005b61ef None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:27.044446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc34373-8188-4a81-8eb2-7ece005b61ef None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:27.044446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc34373-8188-4a81-8eb2-7ece005b61ef None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:27.044446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 598/1174] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:27.049758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a54fb4-ac2a-4b72-a67d-e49ee60ce269 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:27.050190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a54fb4-ac2a-4b72-a67d-e49ee60ce269 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:27.050324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a54fb4-ac2a-4b72-a67d-e49ee60ce269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:27.068106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-36a54fb4-ac2a-4b72-a67d-e49ee60ce269 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:27.069323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-36a54fb4-ac2a-4b72-a67d-e49ee60ce269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:27.079678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:27.134690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 577/1175] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:27.159827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc34373-8188-4a81-8eb2-7ece005b61ef None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:27.256907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc34373-8188-4a81-8eb2-7ece005b61ef None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:27.289986 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130340b-048a-4413-8038-27fd611fda5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:27.290834 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130340b-048a-4413-8038-27fd611fda5a None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:27.291109 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130340b-048a-4413-8038-27fd611fda5a None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:27.291377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130340b-048a-4413-8038-27fd611fda5a None admin] PATH_INFO: `/v3/domains/25c11cfccf2145829cfb233e876d4368` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:27.292633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130340b-048a-4413-8038-27fd611fda5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=25c11cfccf2145829cfb233e876d4368)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:27.293996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130340b-048a-4413-8038-27fd611fda5a None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:27.295622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 599/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Mon Jul 25 08:34:27 2022] GET /identity/v3/domains/25c11cfccf2145829cfb233e876d4368 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:27.301258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a54fb4-ac2a-4b72-a67d-e49ee60ce269 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:27.396008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a54fb4-ac2a-4b72-a67d-e49ee60ce269 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:27.428927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2d4a4-44bd-4185-9406-0a61982b654f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:27.429599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2d4a4-44bd-4185-9406-0a61982b654f None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:27.429882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2d4a4-44bd-4185-9406-0a61982b654f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:27.430157 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2d4a4-44bd-4185-9406-0a61982b654f None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:27.430858 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2d4a4-44bd-4185-9406-0a61982b654f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:27.432270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2d4a4-44bd-4185-9406-0a61982b654f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:27.453374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 578/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:34:27 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:34:29.453217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95992caa-cd85-4a04-ae48-c0cd7616bd4d None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:29.453769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95992caa-cd85-4a04-ae48-c0cd7616bd4d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:29.453769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95992caa-cd85-4a04-ae48-c0cd7616bd4d None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:29.455150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 600/1178] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:29.460238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19c2b0-b3a7-4b36-856d-cb3610750366 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:29.460692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19c2b0-b3a7-4b36-856d-cb3610750366 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:29.460827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19c2b0-b3a7-4b36-856d-cb3610750366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:29.467810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:29.480777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3d19c2b0-b3a7-4b36-856d-cb3610750366 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:29.481958 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3d19c2b0-b3a7-4b36-856d-cb3610750366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:29.546542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 579/1179] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:29.569264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95992caa-cd85-4a04-ae48-c0cd7616bd4d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:29.590729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:29.668448 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95992caa-cd85-4a04-ae48-c0cd7616bd4d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:29.701492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adbcf2-3f89-4063-8dd2-5dcb61ce43d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:29.702258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adbcf2-3f89-4063-8dd2-5dcb61ce43d0 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:29.702537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adbcf2-3f89-4063-8dd2-5dcb61ce43d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:29.702809 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adbcf2-3f89-4063-8dd2-5dcb61ce43d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:29.712462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55adbcf2-3f89-4063-8dd2-5dcb61ce43d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-55adbcf2-3f89-4063-8dd2-5dcb61ce43d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:34:29.720033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:34:29.721385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:34:29.722476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 601/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:34:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:34:29.727089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19c2b0-b3a7-4b36-856d-cb3610750366 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:29.822390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19c2b0-b3a7-4b36-856d-cb3610750366 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:29.850404 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:29.855691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13871df-920f-4702-ba07-f943dd68f902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:29.856348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13871df-920f-4702-ba07-f943dd68f902 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:29.856656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13871df-920f-4702-ba07-f943dd68f902 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:29.856933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13871df-920f-4702-ba07-f943dd68f902 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:29.858264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13871df-920f-4702-ba07-f943dd68f902 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:34:29.858567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13871df-920f-4702-ba07-f943dd68f902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:29.859954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13871df-920f-4702-ba07-f943dd68f902 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:29.867844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 580/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:34:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:29.873338 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adbcf2-3f89-4063-8dd2-5dcb61ce43d0 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:29.951024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:29.969918 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adbcf2-3f89-4063-8dd2-5dcb61ce43d0 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:30.002006 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497789a-d638-41fa-a149-bedb8684e6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:30.002857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497789a-d638-41fa-a149-bedb8684e6dd None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:30.003137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497789a-d638-41fa-a149-bedb8684e6dd None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:30.003412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497789a-d638-41fa-a149-bedb8684e6dd None admin] PATH_INFO: `/v3/users/3816da470a634531a67aa72f0e2d83b2` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:30.016553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497789a-d638-41fa-a149-bedb8684e6dd None admin] RBAC: Authorizing `identity:get_user(user_id=3816da470a634531a67aa72f0e2d83b2)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:30.018004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497789a-d638-41fa-a149-bedb8684e6dd None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:30.019610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 602/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:34:29 2022] GET /identity/v3/users/3816da470a634531a67aa72f0e2d83b2 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:34:30.025453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13871df-920f-4702-ba07-f943dd68f902 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:30.120306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13871df-920f-4702-ba07-f943dd68f902 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:30.152619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d162e-135e-4170-a971-f710ba807ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:30.153252 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d162e-135e-4170-a971-f710ba807ec1 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:30.153534 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d162e-135e-4170-a971-f710ba807ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:30.153805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d162e-135e-4170-a971-f710ba807ec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:30.210893 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:30.218455 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d162e-135e-4170-a971-f710ba807ec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:30.219876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d162e-135e-4170-a971-f710ba807ec1 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:30.314232 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 581/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:34:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:34:30.319981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497789a-d638-41fa-a149-bedb8684e6dd None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:30.415219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497789a-d638-41fa-a149-bedb8684e6dd None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:30.448745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49381df0-aee6-452f-a997-81414db00e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:30.449582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49381df0-aee6-452f-a997-81414db00e4e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:30.449861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49381df0-aee6-452f-a997-81414db00e4e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:30.450131 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49381df0-aee6-452f-a997-81414db00e4e None admin] PATH_INFO: `/v3/projects/53ee2caefd734459b9a9a3368b5727c5` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:30.451314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49381df0-aee6-452f-a997-81414db00e4e None admin] RBAC: Authorizing `identity:get_project(project_id=53ee2caefd734459b9a9a3368b5727c5)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:30.452756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49381df0-aee6-452f-a997-81414db00e4e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:30.454414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 603/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:34:30 2022] GET /identity/v3/projects/53ee2caefd734459b9a9a3368b5727c5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:30.460357 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d162e-135e-4170-a971-f710ba807ec1 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:30.550740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:30.557764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d162e-135e-4170-a971-f710ba807ec1 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:30.591067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70348e41-98bf-4201-b92c-f8019ac2f4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:30.592203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70348e41-98bf-4201-b92c-f8019ac2f4a1 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:30.592488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70348e41-98bf-4201-b92c-f8019ac2f4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:30.592796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70348e41-98bf-4201-b92c-f8019ac2f4a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:30.593630 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70348e41-98bf-4201-b92c-f8019ac2f4a1 None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=53ee2caefd734459b9a9a3368b5727c5, user.id=3816da470a634531a67aa72f0e2d83b2) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:34:30.593940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70348e41-98bf-4201-b92c-f8019ac2f4a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:30.595354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70348e41-98bf-4201-b92c-f8019ac2f4a1 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:30.600693 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 582/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:34:30 2022] GET /identity/v3/role_assignments?user.id=3816da470a634531a67aa72f0e2d83b2&scope.project.id=53ee2caefd734459b9a9a3368b5727c5&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:32.555702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48d321-d817-4d93-b436-1f859829cf16 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:32.556248 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48d321-d817-4d93-b436-1f859829cf16 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:32.556248 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48d321-d817-4d93-b436-1f859829cf16 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:32.557570 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 604/1186] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:34:32.562808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e56781-f891-4784-82fb-4ef01a549b84 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:32.563241 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e56781-f891-4784-82fb-4ef01a549b84 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:32.563369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e56781-f891-4784-82fb-4ef01a549b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:32.581551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-f4e56781-f891-4784-82fb-4ef01a549b84 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:32.582718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f4e56781-f891-4784-82fb-4ef01a549b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:32.646425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 583/1187] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:32.669434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48d321-d817-4d93-b436-1f859829cf16 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:32.678251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:32.770064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48d321-d817-4d93-b436-1f859829cf16 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:32.802500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd0fdc-2b6c-41c3-84e3-599dbf1366bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:32.803284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0fdc-2b6c-41c3-84e3-599dbf1366bd None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:32.803566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0fdc-2b6c-41c3-84e3-599dbf1366bd None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:32.803845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0fdc-2b6c-41c3-84e3-599dbf1366bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:32.814270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd0fdc-2b6c-41c3-84e3-599dbf1366bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-3fcd0fdc-2b6c-41c3-84e3-599dbf1366bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:34:32.821857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:34:32.823004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:34:32.824104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 605/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:34:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:34:32.829476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e56781-f891-4784-82fb-4ef01a549b84 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:32.912313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:32.924275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e56781-f891-4784-82fb-4ef01a549b84 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:32.957285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab22730-1aaf-4bb1-8302-64e67c5ace56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:32.957942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab22730-1aaf-4bb1-8302-64e67c5ace56 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:32.958226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab22730-1aaf-4bb1-8302-64e67c5ace56 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:32.958501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab22730-1aaf-4bb1-8302-64e67c5ace56 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:32.959243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab22730-1aaf-4bb1-8302-64e67c5ace56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:34:32.959549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab22730-1aaf-4bb1-8302-64e67c5ace56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:32.960929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab22730-1aaf-4bb1-8302-64e67c5ace56 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:32.968744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 584/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:34:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:32.974320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd0fdc-2b6c-41c3-84e3-599dbf1366bd None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:33.037869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:33.072183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd0fdc-2b6c-41c3-84e3-599dbf1366bd None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:33.104796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fde12-70a0-4173-bc03-adca47e8f05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:33.105646 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fde12-70a0-4173-bc03-adca47e8f05b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:33.105924 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fde12-70a0-4173-bc03-adca47e8f05b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:33.106194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fde12-70a0-4173-bc03-adca47e8f05b None admin] PATH_INFO: `/v3/users/3816da470a634531a67aa72f0e2d83b2` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:33.107368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fde12-70a0-4173-bc03-adca47e8f05b None admin] RBAC: Authorizing `identity:get_user(user_id=3816da470a634531a67aa72f0e2d83b2)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:33.108815 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fde12-70a0-4173-bc03-adca47e8f05b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:33.110452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 606/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:34:32 2022] GET /identity/v3/users/3816da470a634531a67aa72f0e2d83b2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:33.115974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab22730-1aaf-4bb1-8302-64e67c5ace56 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:33.212145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab22730-1aaf-4bb1-8302-64e67c5ace56 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:33.244189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7cea0-05d4-4121-a3fc-299df29ac44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:33.245057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7cea0-05d4-4121-a3fc-299df29ac44a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:33.245337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7cea0-05d4-4121-a3fc-299df29ac44a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:33.245607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7cea0-05d4-4121-a3fc-299df29ac44a None admin] PATH_INFO: `/v3/projects/53ee2caefd734459b9a9a3368b5727c5` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:33.246759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7cea0-05d4-4121-a3fc-299df29ac44a None admin] RBAC: Authorizing `identity:get_project(project_id=53ee2caefd734459b9a9a3368b5727c5)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:33.248941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7cea0-05d4-4121-a3fc-299df29ac44a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:33.250589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 585/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:34:33 2022] GET /identity/v3/projects/53ee2caefd734459b9a9a3368b5727c5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:33.256204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fde12-70a0-4173-bc03-adca47e8f05b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:33.352790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fde12-70a0-4173-bc03-adca47e8f05b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:33.385663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314bf24-733c-425e-8c92-3195b6f7ad7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:33.386544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314bf24-733c-425e-8c92-3195b6f7ad7b None admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:33.386828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314bf24-733c-425e-8c92-3195b6f7ad7b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:33.387104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314bf24-733c-425e-8c92-3195b6f7ad7b None admin] PATH_INFO: `/v3/projects/53ee2caefd734459b9a9a3368b5727c5/users/3816da470a634531a67aa72f0e2d83b2/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:33.389103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314bf24-733c-425e-8c92-3195b6f7ad7b None admin] RBAC: Authorizing `identity:create_grant(project_id=53ee2caefd734459b9a9a3368b5727c5, user_id=3816da470a634531a67aa72f0e2d83b2, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:33.390516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314bf24-733c-425e-8c92-3195b6f7ad7b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:33.392706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:33.403532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 607/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:34:33 2022] PUT /identity/v3/projects/53ee2caefd734459b9a9a3368b5727c5/users/3816da470a634531a67aa72f0e2d83b2/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:34:35.358896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0477f9-e653-4edb-8e04-104328c7593b None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:35.359361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0477f9-e653-4edb-8e04-104328c7593b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:35.359463 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0477f9-e653-4edb-8e04-104328c7593b None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:35.360872 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 586/1193] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:34:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:34:35.365887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce15ff-7951-45eb-ac1e-c7ed4376ed83 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:35.366284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce15ff-7951-45eb-ac1e-c7ed4376ed83 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:35.366439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce15ff-7951-45eb-ac1e-c7ed4376ed83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:35.384184 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-b6ce15ff-7951-45eb-ac1e-c7ed4376ed83 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:34:35.385398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b6ce15ff-7951-45eb-ac1e-c7ed4376ed83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:34:35.450505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 608/1194] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:34:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:34:35.473998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0477f9-e653-4edb-8e04-104328c7593b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:35.493318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:35.572405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0477f9-e653-4edb-8e04-104328c7593b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:35.605092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c883b1b-5043-471a-98fb-7186de5bd746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:35.605869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c883b1b-5043-471a-98fb-7186de5bd746 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:35.606153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c883b1b-5043-471a-98fb-7186de5bd746 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:35.606430 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c883b1b-5043-471a-98fb-7186de5bd746 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:35.616221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c883b1b-5043-471a-98fb-7186de5bd746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-6c883b1b-5043-471a-98fb-7186de5bd746 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:34:35.623912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:34:35.625031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:34:35.625911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 587/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:34:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:34:35.631188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce15ff-7951-45eb-ac1e-c7ed4376ed83 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:35.727025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce15ff-7951-45eb-ac1e-c7ed4376ed83 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:35.749866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:35.761146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23839492-57bb-475a-8699-18fa3cda37e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:35.761800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23839492-57bb-475a-8699-18fa3cda37e2 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:35.762084 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23839492-57bb-475a-8699-18fa3cda37e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:35.762362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23839492-57bb-475a-8699-18fa3cda37e2 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:35.763101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23839492-57bb-475a-8699-18fa3cda37e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:34:35.763997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23839492-57bb-475a-8699-18fa3cda37e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:35.765382 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23839492-57bb-475a-8699-18fa3cda37e2 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:35.773293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 609/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:34:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:35.779096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c883b1b-5043-471a-98fb-7186de5bd746 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:35.855991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:35.877546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c883b1b-5043-471a-98fb-7186de5bd746 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:35.910241 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57136880-2dec-4996-8bb8-1f4cfc81833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:35.911099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57136880-2dec-4996-8bb8-1f4cfc81833b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:35.911385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57136880-2dec-4996-8bb8-1f4cfc81833b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:35.911664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57136880-2dec-4996-8bb8-1f4cfc81833b None admin] PATH_INFO: `/v3/users/3816da470a634531a67aa72f0e2d83b2` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:35.912891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57136880-2dec-4996-8bb8-1f4cfc81833b None admin] RBAC: Authorizing `identity:get_user(user_id=3816da470a634531a67aa72f0e2d83b2)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:35.914327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57136880-2dec-4996-8bb8-1f4cfc81833b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:35.915943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 588/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Jul 25 08:34:35 2022] GET /identity/v3/users/3816da470a634531a67aa72f0e2d83b2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:35.921468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23839492-57bb-475a-8699-18fa3cda37e2 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:36.018470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23839492-57bb-475a-8699-18fa3cda37e2 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:36.050585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691f881-12d4-41c8-802c-4606d3d5485e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:36.051215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f881-12d4-41c8-802c-4606d3d5485e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:36.051493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f881-12d4-41c8-802c-4606d3d5485e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:36.051764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f881-12d4-41c8-802c-4606d3d5485e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:36.103458 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:36.116455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691f881-12d4-41c8-802c-4606d3d5485e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:36.117866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691f881-12d4-41c8-802c-4606d3d5485e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:36.212186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 610/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:34:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:34:36.217940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57136880-2dec-4996-8bb8-1f4cfc81833b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:36.315403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57136880-2dec-4996-8bb8-1f4cfc81833b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:36.346796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32380f-a1ba-439c-ac0a-805d765ae46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:36.347642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32380f-a1ba-439c-ac0a-805d765ae46e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:36.347926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32380f-a1ba-439c-ac0a-805d765ae46e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:36.348201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32380f-a1ba-439c-ac0a-805d765ae46e None admin] PATH_INFO: `/v3/projects/53ee2caefd734459b9a9a3368b5727c5` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:36.349383 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32380f-a1ba-439c-ac0a-805d765ae46e None admin] RBAC: Authorizing `identity:get_project(project_id=53ee2caefd734459b9a9a3368b5727c5)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:36.350814 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32380f-a1ba-439c-ac0a-805d765ae46e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:36.352488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 589/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Jul 25 08:34:36 2022] GET /identity/v3/projects/53ee2caefd734459b9a9a3368b5727c5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:36.358952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691f881-12d4-41c8-802c-4606d3d5485e None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:34:36.440255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:36.454742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691f881-12d4-41c8-802c-4606d3d5485e None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:34:36.487183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc55ec-dc8f-4be3-b408-67422ceb2b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:34:36.487818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc55ec-dc8f-4be3-b408-67422ceb2b47 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:34:36.488101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc55ec-dc8f-4be3-b408-67422ceb2b47 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:34:36.488375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc55ec-dc8f-4be3-b408-67422ceb2b47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:34:36.489221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc55ec-dc8f-4be3-b408-67422ceb2b47 None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=53ee2caefd734459b9a9a3368b5727c5, user.id=3816da470a634531a67aa72f0e2d83b2) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:34:36.489529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc55ec-dc8f-4be3-b408-67422ceb2b47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:34:36.490931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc55ec-dc8f-4be3-b408-67422ceb2b47 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:34:36.496539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 611/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:34:36 2022] GET /identity/v3/role_assignments?user.id=3816da470a634531a67aa72f0e2d83b2&scope.project.id=53ee2caefd734459b9a9a3368b5727c5&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:34:46.342756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=93 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:34:46.493996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=25 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:31.085973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d85f9-f791-4f6f-9a30-f9c16b8b076a None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:31.086579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d85f9-f791-4f6f-9a30-f9c16b8b076a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:31.086579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d85f9-f791-4f6f-9a30-f9c16b8b076a None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:31.087908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 590/1201] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:35:31.093861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d9492-9050-439d-ba2a-e60d16e7d062 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:31.094308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d9492-9050-439d-ba2a-e60d16e7d062 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:31.094418 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d9492-9050-439d-ba2a-e60d16e7d062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:31.114325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c39d9492-9050-439d-ba2a-e60d16e7d062 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:35:31.115879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c39d9492-9050-439d-ba2a-e60d16e7d062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:35:31.194543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 612/1202] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:35:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:35:31.222868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d85f9-f791-4f6f-9a30-f9c16b8b076a None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:31.350958 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d85f9-f791-4f6f-9a30-f9c16b8b076a None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:31.394417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af78b0-6b5b-4b16-b2fe-3c650f6984cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:31.395229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af78b0-6b5b-4b16-b2fe-3c650f6984cd None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:31.395561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af78b0-6b5b-4b16-b2fe-3c650f6984cd None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:31.395889 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af78b0-6b5b-4b16-b2fe-3c650f6984cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:31.408467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af78b0-6b5b-4b16-b2fe-3c650f6984cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:31.410121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af78b0-6b5b-4b16-b2fe-3c650f6984cd None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-d5af78b0-6b5b-4b16-b2fe-3c650f6984cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:31.431182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:31.432825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:31.433865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:31.433865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:31.433865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:31.433865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:31.433865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 591/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:31.439376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d9492-9050-439d-ba2a-e60d16e7d062 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:31.570428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d9492-9050-439d-ba2a-e60d16e7d062 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:31.611999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:31.620365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba395b87-488b-4b2d-906a-612a530130c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:31.621168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba395b87-488b-4b2d-906a-612a530130c0 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:31.621482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba395b87-488b-4b2d-906a-612a530130c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:31.621862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba395b87-488b-4b2d-906a-612a530130c0 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:31.622749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba395b87-488b-4b2d-906a-612a530130c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:31.623103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba395b87-488b-4b2d-906a-612a530130c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:31.624698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba395b87-488b-4b2d-906a-612a530130c0 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:31.639226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 613/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:31.645842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af78b0-6b5b-4b16-b2fe-3c650f6984cd None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:31.713440 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:31.770239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af78b0-6b5b-4b16-b2fe-3c650f6984cd None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:31.817783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6903ad-65f3-4cb0-ab32-fe7250702afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:31.818471 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6903ad-65f3-4cb0-ab32-fe7250702afe None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:31.818774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6903ad-65f3-4cb0-ab32-fe7250702afe None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:31.819031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6903ad-65f3-4cb0-ab32-fe7250702afe None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:31.819886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6903ad-65f3-4cb0-ab32-fe7250702afe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:31.821308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6903ad-65f3-4cb0-ab32-fe7250702afe None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:31.859843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 592/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Jul 25 08:35:31 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:35:34.078347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d01eb5-e49a-4c4e-a943-f099ccdc0a22 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:34.078950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d01eb5-e49a-4c4e-a943-f099ccdc0a22 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:34.078950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d01eb5-e49a-4c4e-a943-f099ccdc0a22 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:34.080426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 614/1206] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:35:34.086580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029010c-96fd-4d4b-a34a-14077159c13e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:34.087050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029010c-96fd-4d4b-a34a-14077159c13e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:34.087322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029010c-96fd-4d4b-a34a-14077159c13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:34.108092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5029010c-96fd-4d4b-a34a-14077159c13e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:35:34.109365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5029010c-96fd-4d4b-a34a-14077159c13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:35:34.189209 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 593/1207] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:35:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:35:34.215141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d01eb5-e49a-4c4e-a943-f099ccdc0a22 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:34.347268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d01eb5-e49a-4c4e-a943-f099ccdc0a22 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:34.389156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dc1fa-e3c0-4cd6-9709-01cb10a45d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:34.390029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dc1fa-e3c0-4cd6-9709-01cb10a45d1b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:34.390316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dc1fa-e3c0-4cd6-9709-01cb10a45d1b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:34.390688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dc1fa-e3c0-4cd6-9709-01cb10a45d1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:34.402233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36dc1fa-e3c0-4cd6-9709-01cb10a45d1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-f36dc1fa-e3c0-4cd6-9709-01cb10a45d1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:35:34.410961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:34.412233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:34.413300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 615/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:35:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:34.418809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029010c-96fd-4d4b-a34a-14077159c13e None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:34.465181 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:34.544366 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029010c-96fd-4d4b-a34a-14077159c13e None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:34.586172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4846d3-ebf5-4dc1-90e1-5b3adc0bfaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:34.586854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4846d3-ebf5-4dc1-90e1-5b3adc0bfaef None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:34.587140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4846d3-ebf5-4dc1-90e1-5b3adc0bfaef None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:34.587419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4846d3-ebf5-4dc1-90e1-5b3adc0bfaef None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:34.588748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4846d3-ebf5-4dc1-90e1-5b3adc0bfaef None admin] RBAC: Adding query filter params (name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:34.589058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4846d3-ebf5-4dc1-90e1-5b3adc0bfaef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:34.590648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4846d3-ebf5-4dc1-90e1-5b3adc0bfaef None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:34.604725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 594/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:35:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:34.607067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dc1fa-e3c0-4cd6-9709-01cb10a45d1b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:34.668499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:34.734969 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dc1fa-e3c0-4cd6-9709-01cb10a45d1b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:34.781331 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82315f42-fd84-4d4e-bd10-3f3d0bc2241b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:34.782296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82315f42-fd84-4d4e-bd10-3f3d0bc2241b None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:34.782586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82315f42-fd84-4d4e-bd10-3f3d0bc2241b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:34.783157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82315f42-fd84-4d4e-bd10-3f3d0bc2241b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:34.796095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82315f42-fd84-4d4e-bd10-3f3d0bc2241b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:34.797801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82315f42-fd84-4d4e-bd10-3f3d0bc2241b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-82315f42-fd84-4d4e-bd10-3f3d0bc2241b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:34.816511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:34.817607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:34.818503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:34.818503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:34.818503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:34.818503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:34.818503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 616/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:34.824063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4846d3-ebf5-4dc1-90e1-5b3adc0bfaef None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:34.946681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:34.950247 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4846d3-ebf5-4dc1-90e1-5b3adc0bfaef None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:34.989532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34fe2a-2f6c-4e83-b57e-89edfdf2d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:34.990903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34fe2a-2f6c-4e83-b57e-89edfdf2d23e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:34.991235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34fe2a-2f6c-4e83-b57e-89edfdf2d23e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:34.991523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34fe2a-2f6c-4e83-b57e-89edfdf2d23e None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:34.992364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34fe2a-2f6c-4e83-b57e-89edfdf2d23e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:34.992710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34fe2a-2f6c-4e83-b57e-89edfdf2d23e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:34.994777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34fe2a-2f6c-4e83-b57e-89edfdf2d23e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:35.007681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 595/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:35.013947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82315f42-fd84-4d4e-bd10-3f3d0bc2241b None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:35.147489 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82315f42-fd84-4d4e-bd10-3f3d0bc2241b None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:35.149223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:35.188452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844af77-77a6-4a57-af38-31928465071e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:35.189287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844af77-77a6-4a57-af38-31928465071e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:35.189568 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844af77-77a6-4a57-af38-31928465071e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:35.189846 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844af77-77a6-4a57-af38-31928465071e None admin] PATH_INFO: `/v3/users/placement` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:35.199369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844af77-77a6-4a57-af38-31928465071e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:35.201142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844af77-77a6-4a57-af38-31928465071e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-6844af77-77a6-4a57-af38-31928465071e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:35:35.219139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:35.220287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:35.221408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:35.221408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:35.221408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:35:35.221408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:35:35.221408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:35:35.221408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:35:35.221408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:35:35.221408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:35:35.221408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 08:35:35.221408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:35.222721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 617/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:35:35.229025 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34fe2a-2f6c-4e83-b57e-89edfdf2d23e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:35.361082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34fe2a-2f6c-4e83-b57e-89edfdf2d23e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:35.404422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d929953-236b-4995-a29e-b869827331ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:35.405219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d929953-236b-4995-a29e-b869827331ba None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:35.405505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d929953-236b-4995-a29e-b869827331ba None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:35.405778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d929953-236b-4995-a29e-b869827331ba None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:35.406626 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d929953-236b-4995-a29e-b869827331ba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:35.406931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d929953-236b-4995-a29e-b869827331ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:35.408759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d929953-236b-4995-a29e-b869827331ba None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:35.429625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 596/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Jul 25 08:35:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:35:35.436129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844af77-77a6-4a57-af38-31928465071e None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:35.562973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844af77-77a6-4a57-af38-31928465071e None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:35.608878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8e153-cfbb-4a0d-bfc0-ee2842ab0571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:35.609550 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8e153-cfbb-4a0d-bfc0-ee2842ab0571 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:35.609823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8e153-cfbb-4a0d-bfc0-ee2842ab0571 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:35.610106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8e153-cfbb-4a0d-bfc0-ee2842ab0571 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:35.648191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:35.691004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8e153-cfbb-4a0d-bfc0-ee2842ab0571 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:35.692479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8e153-cfbb-4a0d-bfc0-ee2842ab0571 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:35.815406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 618/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:35:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:35:35.820949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d929953-236b-4995-a29e-b869827331ba None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:35.841756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:35.933245 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d929953-236b-4995-a29e-b869827331ba None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:35.974378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13249aa2-ac10-4fc2-b5d1-992a34d0f37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:35.975399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13249aa2-ac10-4fc2-b5d1-992a34d0f37b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:35.975684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13249aa2-ac10-4fc2-b5d1-992a34d0f37b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:35.976026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13249aa2-ac10-4fc2-b5d1-992a34d0f37b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:35.988390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13249aa2-ac10-4fc2-b5d1-992a34d0f37b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:35.990184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13249aa2-ac10-4fc2-b5d1-992a34d0f37b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-13249aa2-ac10-4fc2-b5d1-992a34d0f37b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:36.011343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:36.012512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:36.013467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:36.013467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:36.013467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:36.013467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:36.013467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 597/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:36.018683 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8e153-cfbb-4a0d-bfc0-ee2842ab0571 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:36.098467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:36.141896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8e153-cfbb-4a0d-bfc0-ee2842ab0571 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:36.182239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dd278-1faf-4133-b90c-67324a49eebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:36.182916 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dd278-1faf-4133-b90c-67324a49eebb None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:36.183234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dd278-1faf-4133-b90c-67324a49eebb None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:36.183439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dd278-1faf-4133-b90c-67324a49eebb None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:36.184195 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dd278-1faf-4133-b90c-67324a49eebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:36.184506 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dd278-1faf-4133-b90c-67324a49eebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:36.186069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dd278-1faf-4133-b90c-67324a49eebb None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:36.205458 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 619/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:36.206800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13249aa2-ac10-4fc2-b5d1-992a34d0f37b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:36.286150 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:36.320864 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13249aa2-ac10-4fc2-b5d1-992a34d0f37b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:36.359861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0205899-b069-4d0b-bbfb-bd628d3ed3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:36.360600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0205899-b069-4d0b-bbfb-bd628d3ed3eb None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:36.360891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0205899-b069-4d0b-bbfb-bd628d3ed3eb None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:36.361174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0205899-b069-4d0b-bbfb-bd628d3ed3eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:36.368049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0205899-b069-4d0b-bbfb-bd628d3ed3eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:36.369574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0205899-b069-4d0b-bbfb-bd628d3ed3eb None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-e0205899-b069-4d0b-bbfb-bd628d3ed3eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:35:36.384177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:36.385871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:36.386968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:35:36.386968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:35:36.386968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:35:36.386968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:35:36.386968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:35:36.386968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:35:36.386968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:35:36.386968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:36.391103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 598/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:35:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:36.393959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dd278-1faf-4133-b90c-67324a49eebb None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:36.513367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dd278-1faf-4133-b90c-67324a49eebb None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:36.537997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:36.554737 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ff12c-73e2-4e93-b389-5d45c4b24b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:36.555511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ff12c-73e2-4e93-b389-5d45c4b24b34 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:36.555810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ff12c-73e2-4e93-b389-5d45c4b24b34 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:36.556150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ff12c-73e2-4e93-b389-5d45c4b24b34 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:36.557668 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ff12c-73e2-4e93-b389-5d45c4b24b34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:36.558100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ff12c-73e2-4e93-b389-5d45c4b24b34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:36.559687 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ff12c-73e2-4e93-b389-5d45c4b24b34 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:36.575701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 620/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:35:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:35:36.582511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0205899-b069-4d0b-bbfb-bd628d3ed3eb None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:36.701362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0205899-b069-4d0b-bbfb-bd628d3ed3eb None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:36.732913 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:36.740234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702aa2f-5f2b-4768-bb00-ab0d1233f42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:36.741047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702aa2f-5f2b-4768-bb00-ab0d1233f42a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:36.741337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702aa2f-5f2b-4768-bb00-ab0d1233f42a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:36.741705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702aa2f-5f2b-4768-bb00-ab0d1233f42a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:36.742724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702aa2f-5f2b-4768-bb00-ab0d1233f42a None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=62bf4ef4fce74f7b91e453f73430d9ae) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:36.743041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702aa2f-5f2b-4768-bb00-ab0d1233f42a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:36.745248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702aa2f-5f2b-4768-bb00-ab0d1233f42a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:36.760667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 599/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:35:36 2022] GET /identity/v3/role_assignments?user.id=62bf4ef4fce74f7b91e453f73430d9ae&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:38.907307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d33218-7911-4c24-b958-6cdbdc134a3d None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:38.907865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d33218-7911-4c24-b958-6cdbdc134a3d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:38.907865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d33218-7911-4c24-b958-6cdbdc134a3d None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:38.909290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 621/1220] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:35:38.914521 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93eb87-10a4-405f-8c6f-18efaa1d7f3d None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:38.914973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93eb87-10a4-405f-8c6f-18efaa1d7f3d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:38.915109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93eb87-10a4-405f-8c6f-18efaa1d7f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:38.934214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-4f93eb87-10a4-405f-8c6f-18efaa1d7f3d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:35:38.935516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4f93eb87-10a4-405f-8c6f-18efaa1d7f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:35:39.021725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 600/1221] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:35:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:35:39.048210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d33218-7911-4c24-b958-6cdbdc134a3d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:39.165113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d33218-7911-4c24-b958-6cdbdc134a3d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:39.204878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72e952-81a9-4a93-a976-f1b65663bf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:39.206008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72e952-81a9-4a93-a976-f1b65663bf08 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:39.206476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72e952-81a9-4a93-a976-f1b65663bf08 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:39.206940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72e952-81a9-4a93-a976-f1b65663bf08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:39.217250 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72e952-81a9-4a93-a976-f1b65663bf08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-3a72e952-81a9-4a93-a976-f1b65663bf08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:35:39.224859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:39.225997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:39.226856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 622/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:35:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:39.232502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93eb87-10a4-405f-8c6f-18efaa1d7f3d None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:39.344153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93eb87-10a4-405f-8c6f-18efaa1d7f3d None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:39.368988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:39.380768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fb8ef-c2fa-47ed-b1ea-f445ec4cbf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:39.381512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fb8ef-c2fa-47ed-b1ea-f445ec4cbf60 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:39.381774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fb8ef-c2fa-47ed-b1ea-f445ec4cbf60 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:39.382091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fb8ef-c2fa-47ed-b1ea-f445ec4cbf60 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:39.382944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fb8ef-c2fa-47ed-b1ea-f445ec4cbf60 None admin] RBAC: Adding query filter params (name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:39.383269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fb8ef-c2fa-47ed-b1ea-f445ec4cbf60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:39.385277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fb8ef-c2fa-47ed-b1ea-f445ec4cbf60 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:39.394282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 601/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:35:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:39.401058 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72e952-81a9-4a93-a976-f1b65663bf08 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:39.446467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:39.506273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72e952-81a9-4a93-a976-f1b65663bf08 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:39.542482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384d5fc-8375-4235-82ad-0ac08f105582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:39.543278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384d5fc-8375-4235-82ad-0ac08f105582 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:39.543565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384d5fc-8375-4235-82ad-0ac08f105582 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:39.543844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384d5fc-8375-4235-82ad-0ac08f105582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:39.554251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384d5fc-8375-4235-82ad-0ac08f105582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:39.555707 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384d5fc-8375-4235-82ad-0ac08f105582 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-b384d5fc-8375-4235-82ad-0ac08f105582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:39.573765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:39.574890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:39.575927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:39.575927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:39.575927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:39.575927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:39.575927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 623/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:39.580856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fb8ef-c2fa-47ed-b1ea-f445ec4cbf60 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:39.683339 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fb8ef-c2fa-47ed-b1ea-f445ec4cbf60 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:39.717603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f4ec-c9d5-44fe-bd8f-74d296d3786a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:39.718308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f4ec-c9d5-44fe-bd8f-74d296d3786a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:39.724351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f4ec-c9d5-44fe-bd8f-74d296d3786a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:39.724730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f4ec-c9d5-44fe-bd8f-74d296d3786a None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:39.725938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627f4ec-c9d5-44fe-bd8f-74d296d3786a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:39.726262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627f4ec-c9d5-44fe-bd8f-74d296d3786a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:39.727641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627f4ec-c9d5-44fe-bd8f-74d296d3786a None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:39.739886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 602/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:39.746039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384d5fc-8375-4235-82ad-0ac08f105582 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:39.850356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:39.857977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384d5fc-8375-4235-82ad-0ac08f105582 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:39.898113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cf56b-a2b7-4f7c-ae39-8309d057fdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:39.899239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cf56b-a2b7-4f7c-ae39-8309d057fdf7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:39.899797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cf56b-a2b7-4f7c-ae39-8309d057fdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:39.900296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cf56b-a2b7-4f7c-ae39-8309d057fdf7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:39.913194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cf56b-a2b7-4f7c-ae39-8309d057fdf7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:39.915452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cf56b-a2b7-4f7c-ae39-8309d057fdf7 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-8b4cf56b-a2b7-4f7c-ae39-8309d057fdf7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:35:39.932477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:39.933661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:39.935095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:39.935095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:39.935095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:35:39.935095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:35:39.935095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:35:39.935095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:35:39.935095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:35:39.935095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:35:39.935095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 08:35:39.935095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:39.935095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 624/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:39.939731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f4ec-c9d5-44fe-bd8f-74d296d3786a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:39.990155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:40.049694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f4ec-c9d5-44fe-bd8f-74d296d3786a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:40.094776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c8d04-d009-4f89-950c-02c61d84fa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:40.095454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c8d04-d009-4f89-950c-02c61d84fa8d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:40.095738 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c8d04-d009-4f89-950c-02c61d84fa8d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:40.096013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c8d04-d009-4f89-950c-02c61d84fa8d None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:40.096983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c8d04-d009-4f89-950c-02c61d84fa8d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:40.097346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c8d04-d009-4f89-950c-02c61d84fa8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:40.098937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c8d04-d009-4f89-950c-02c61d84fa8d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:40.118408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 603/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Jul 25 08:35:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:40.124939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cf56b-a2b7-4f7c-ae39-8309d057fdf7 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:40.234810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cf56b-a2b7-4f7c-ae39-8309d057fdf7 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:40.270823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f251b3-9347-45a7-99f2-f3d238c9f8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:40.271920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f251b3-9347-45a7-99f2-f3d238c9f8bc None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:40.272229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f251b3-9347-45a7-99f2-f3d238c9f8bc None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:40.272540 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f251b3-9347-45a7-99f2-f3d238c9f8bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:40.284856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f251b3-9347-45a7-99f2-f3d238c9f8bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:40.286306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f251b3-9347-45a7-99f2-f3d238c9f8bc None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-c1f251b3-9347-45a7-99f2-f3d238c9f8bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:40.307500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:40.308856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:40.310165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:40.310165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:40.310165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:40.310165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:40.310165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 625/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:35:40.314914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c8d04-d009-4f89-950c-02c61d84fa8d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:40.428917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c8d04-d009-4f89-950c-02c61d84fa8d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:40.432134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:40.467827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce126207-f5a9-41f1-9112-b7ff9938b70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:40.468527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce126207-f5a9-41f1-9112-b7ff9938b70d None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:40.468848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce126207-f5a9-41f1-9112-b7ff9938b70d None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:40.469118 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce126207-f5a9-41f1-9112-b7ff9938b70d None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:40.469846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce126207-f5a9-41f1-9112-b7ff9938b70d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:40.470153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce126207-f5a9-41f1-9112-b7ff9938b70d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:40.471565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce126207-f5a9-41f1-9112-b7ff9938b70d None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:40.485987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 604/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:40.491987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f251b3-9347-45a7-99f2-f3d238c9f8bc None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:40.501715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:40.602216 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f251b3-9347-45a7-99f2-f3d238c9f8bc None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:40.641870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d822379-c745-45f1-9670-f8acde3bf956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:40.642789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d822379-c745-45f1-9670-f8acde3bf956 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:40.643136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d822379-c745-45f1-9670-f8acde3bf956 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:40.643393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d822379-c745-45f1-9670-f8acde3bf956 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:40.652175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d822379-c745-45f1-9670-f8acde3bf956 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:40.653918 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d822379-c745-45f1-9670-f8acde3bf956 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-9d822379-c745-45f1-9670-f8acde3bf956 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:35:40.668879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:40.669806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:40.670770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:35:40.670770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:35:40.670770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:35:40.670770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:35:40.670770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:35:40.670770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:35:40.670770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:35:40.670770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:40.670770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 626/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:35:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:40.675873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce126207-f5a9-41f1-9112-b7ff9938b70d None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:40.784217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce126207-f5a9-41f1-9112-b7ff9938b70d None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:40.822530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b70da-92cd-4701-adb7-c9eded3c59e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:40.823262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b70da-92cd-4701-adb7-c9eded3c59e0 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:40.823581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b70da-92cd-4701-adb7-c9eded3c59e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:40.823877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b70da-92cd-4701-adb7-c9eded3c59e0 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:40.824898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b70da-92cd-4701-adb7-c9eded3c59e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:40.825314 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b70da-92cd-4701-adb7-c9eded3c59e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:40.827047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b70da-92cd-4701-adb7-c9eded3c59e0 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:40.840951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 605/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:35:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:40.847554 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d822379-c745-45f1-9670-f8acde3bf956 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:40.919572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:40.958347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d822379-c745-45f1-9670-f8acde3bf956 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:40.996163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7b33a-dc71-4049-a219-a43a7567487b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:40.997096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7b33a-dc71-4049-a219-a43a7567487b None admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:40.997386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7b33a-dc71-4049-a219-a43a7567487b None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:40.997673 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7b33a-dc71-4049-a219-a43a7567487b None admin] PATH_INFO: `/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/62bf4ef4fce74f7b91e453f73430d9ae/roles/45565d93c8c142e086e6b7ce62ff84e4` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:41.012523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7b33a-dc71-4049-a219-a43a7567487b None admin] RBAC: Authorizing `identity:create_grant(project_id=2ee27c4da77c47428e02fd65e890d256, user_id=62bf4ef4fce74f7b91e453f73430d9ae, role_id=45565d93c8c142e086e6b7ce62ff84e4)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:41.014009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7b33a-dc71-4049-a219-a43a7567487b None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:41.027299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 627/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:35:40 2022] PUT /identity/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/62bf4ef4fce74f7b91e453f73430d9ae/roles/45565d93c8c142e086e6b7ce62ff84e4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:35:43.113424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec25eb-d7a4-4f91-aa92-03308e442103 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:43.113979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec25eb-d7a4-4f91-aa92-03308e442103 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:43.113979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec25eb-d7a4-4f91-aa92-03308e442103 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:43.115371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 606/1233] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:35:43.120742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52008ec-dbb6-4968-bfcc-1986e6749294 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:43.121277 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52008ec-dbb6-4968-bfcc-1986e6749294 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:43.121420 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52008ec-dbb6-4968-bfcc-1986e6749294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:43.140784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a52008ec-dbb6-4968-bfcc-1986e6749294 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:35:43.141983 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a52008ec-dbb6-4968-bfcc-1986e6749294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:35:43.220602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 628/1234] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:35:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:35:43.237655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec25eb-d7a4-4f91-aa92-03308e442103 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:43.264655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:43.352032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec25eb-d7a4-4f91-aa92-03308e442103 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:43.389286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40ad81-9bc7-4e1b-99c9-f6646a7ebafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:43.390127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40ad81-9bc7-4e1b-99c9-f6646a7ebafa None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:43.390391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40ad81-9bc7-4e1b-99c9-f6646a7ebafa None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:43.390621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40ad81-9bc7-4e1b-99c9-f6646a7ebafa None admin] PATH_INFO: `/v3/roles/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:43.401723 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40ad81-9bc7-4e1b-99c9-f6646a7ebafa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-6d40ad81-9bc7-4e1b-99c9-f6646a7ebafa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:35:43.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:43.410553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:43.411730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 607/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Jul 25 08:35:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:43.417164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52008ec-dbb6-4968-bfcc-1986e6749294 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:43.468873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:43.526920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52008ec-dbb6-4968-bfcc-1986e6749294 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:43.561652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824ddac-14ff-46db-ae14-cc2dbed50c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:43.562397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824ddac-14ff-46db-ae14-cc2dbed50c0f None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:43.562725 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824ddac-14ff-46db-ae14-cc2dbed50c0f None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:43.563003 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824ddac-14ff-46db-ae14-cc2dbed50c0f None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:43.563751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824ddac-14ff-46db-ae14-cc2dbed50c0f None admin] RBAC: Adding query filter params (name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:43.564059 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824ddac-14ff-46db-ae14-cc2dbed50c0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:43.565469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824ddac-14ff-46db-ae14-cc2dbed50c0f None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:43.574312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 629/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Jul 25 08:35:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:35:43.580661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40ad81-9bc7-4e1b-99c9-f6646a7ebafa None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:43.672979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:43.690441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40ad81-9bc7-4e1b-99c9-f6646a7ebafa None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:43.730321 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f56cc4-8d99-4dc0-8847-bd9bb8d5aaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:43.731341 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f56cc4-8d99-4dc0-8847-bd9bb8d5aaba None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:43.731631 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f56cc4-8d99-4dc0-8847-bd9bb8d5aaba None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:43.731906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f56cc4-8d99-4dc0-8847-bd9bb8d5aaba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:43.745237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f56cc4-8d99-4dc0-8847-bd9bb8d5aaba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:43.746694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f56cc4-8d99-4dc0-8847-bd9bb8d5aaba None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-89f56cc4-8d99-4dc0-8847-bd9bb8d5aaba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:43.775373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:43.776527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:43.777629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:43.777629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:43.777629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:43.777629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:43.777629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 608/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:43.782979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824ddac-14ff-46db-ae14-cc2dbed50c0f None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:43.890426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824ddac-14ff-46db-ae14-cc2dbed50c0f None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:43.900371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:43.929662 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4e778-ae3b-4c43-a473-d068ab3ef921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:43.930455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4e778-ae3b-4c43-a473-d068ab3ef921 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:43.930744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4e778-ae3b-4c43-a473-d068ab3ef921 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:43.931181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4e778-ae3b-4c43-a473-d068ab3ef921 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:43.932869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4e778-ae3b-4c43-a473-d068ab3ef921 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:43.933233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4e778-ae3b-4c43-a473-d068ab3ef921 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:43.934942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4e778-ae3b-4c43-a473-d068ab3ef921 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:43.948253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 630/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:43.953948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f56cc4-8d99-4dc0-8847-bd9bb8d5aaba None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:44.072601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f56cc4-8d99-4dc0-8847-bd9bb8d5aaba None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:44.116334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2b3ef-9af9-4437-b725-839f4888506f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:44.117279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2b3ef-9af9-4437-b725-839f4888506f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:44.117567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2b3ef-9af9-4437-b725-839f4888506f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:44.117843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2b3ef-9af9-4437-b725-839f4888506f None admin] PATH_INFO: `/v3/users/placement` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:44.121006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:44.128145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2b3ef-9af9-4437-b725-839f4888506f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:44.129705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2b3ef-9af9-4437-b725-839f4888506f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-38f2b3ef-9af9-4437-b725-839f4888506f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:35:44.144549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:44.145669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:44.146896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:44.146896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:44.146896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:35:44.146896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:35:44.146896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:35:44.146896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:35:44.146896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:35:44.146896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:35:44.146896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 08:35:44.146896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:44.146896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 609/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:44.152279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4e778-ae3b-4c43-a473-d068ab3ef921 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:44.254687 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4e778-ae3b-4c43-a473-d068ab3ef921 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:44.294393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640de89-3ede-4047-b5b3-0438bfb25d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:44.295076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640de89-3ede-4047-b5b3-0438bfb25d8e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:44.295357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640de89-3ede-4047-b5b3-0438bfb25d8e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:44.295630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640de89-3ede-4047-b5b3-0438bfb25d8e None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:44.296477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640de89-3ede-4047-b5b3-0438bfb25d8e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:44.296810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640de89-3ede-4047-b5b3-0438bfb25d8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:44.298247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640de89-3ede-4047-b5b3-0438bfb25d8e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:44.317119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 631/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Jul 25 08:35:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:44.322879 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2b3ef-9af9-4437-b725-839f4888506f None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:44.428207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2b3ef-9af9-4437-b725-839f4888506f None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:44.465862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5d329-8f91-4de9-8316-3f708ce4aace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:44.466523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5d329-8f91-4de9-8316-3f708ce4aace None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:44.466804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5d329-8f91-4de9-8316-3f708ce4aace None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:44.467084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5d329-8f91-4de9-8316-3f708ce4aace None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:44.530877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:44.538193 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5d329-8f91-4de9-8316-3f708ce4aace None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:44.539683 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5d329-8f91-4de9-8316-3f708ce4aace None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:44.648584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 610/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:35:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:35:44.655125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640de89-3ede-4047-b5b3-0438bfb25d8e None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:44.684596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:44.776050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640de89-3ede-4047-b5b3-0438bfb25d8e None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:44.816500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7848d8-e8d2-43bc-983f-a7a5a87c7429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:44.817622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7848d8-e8d2-43bc-983f-a7a5a87c7429 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:44.818048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7848d8-e8d2-43bc-983f-a7a5a87c7429 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:44.818376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7848d8-e8d2-43bc-983f-a7a5a87c7429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:44.831004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7848d8-e8d2-43bc-983f-a7a5a87c7429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:44.832633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7848d8-e8d2-43bc-983f-a7a5a87c7429 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-ba7848d8-e8d2-43bc-983f-a7a5a87c7429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:44.851857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:44.852940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:44.853930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:44.853930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:44.853930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:44.853930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:44.853930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 632/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:44.859934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5d329-8f91-4de9-8316-3f708ce4aace None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:44.964279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:44.970950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5d329-8f91-4de9-8316-3f708ce4aace None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:45.011612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3000f-4d7c-42c8-8304-45bfa5f04a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:45.012293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3000f-4d7c-42c8-8304-45bfa5f04a21 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:45.012600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3000f-4d7c-42c8-8304-45bfa5f04a21 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:45.013055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3000f-4d7c-42c8-8304-45bfa5f04a21 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:45.013792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3000f-4d7c-42c8-8304-45bfa5f04a21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:45.014107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3000f-4d7c-42c8-8304-45bfa5f04a21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:45.015589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3000f-4d7c-42c8-8304-45bfa5f04a21 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:45.027741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 611/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:45.034142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7848d8-e8d2-43bc-983f-a7a5a87c7429 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:45.131294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:45.156148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7848d8-e8d2-43bc-983f-a7a5a87c7429 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:45.197354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7cfde-0d04-4018-92e9-8b8efb3b13a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:45.198374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7cfde-0d04-4018-92e9-8b8efb3b13a9 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:45.198808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7cfde-0d04-4018-92e9-8b8efb3b13a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:45.199204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7cfde-0d04-4018-92e9-8b8efb3b13a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:45.207087 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7cfde-0d04-4018-92e9-8b8efb3b13a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:45.208907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7cfde-0d04-4018-92e9-8b8efb3b13a9 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-a7f7cfde-0d04-4018-92e9-8b8efb3b13a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:35:45.226462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:45.227879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:45.229333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:35:45.229333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:35:45.229333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:35:45.229333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:35:45.229333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:35:45.229333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:35:45.229333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:35:45.229333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:45.229333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 633/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:35:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:45.234460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3000f-4d7c-42c8-8304-45bfa5f04a21 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:45.353813 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3000f-4d7c-42c8-8304-45bfa5f04a21 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:45.392690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ea793-153e-4dcb-a41e-80dbb60fc873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:45.393371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ea793-153e-4dcb-a41e-80dbb60fc873 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:45.393697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ea793-153e-4dcb-a41e-80dbb60fc873 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:45.393964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ea793-153e-4dcb-a41e-80dbb60fc873 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:45.394763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ea793-153e-4dcb-a41e-80dbb60fc873 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:45.395097 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ea793-153e-4dcb-a41e-80dbb60fc873 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:45.396776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ea793-153e-4dcb-a41e-80dbb60fc873 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:45.410072 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 612/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:35:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:45.416648 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7cfde-0d04-4018-92e9-8b8efb3b13a9 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:45.525812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7cfde-0d04-4018-92e9-8b8efb3b13a9 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:45.566909 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d1c30-4d4a-492e-892d-a179c9e1a496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:45.567586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d1c30-4d4a-492e-892d-a179c9e1a496 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:45.567867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d1c30-4d4a-492e-892d-a179c9e1a496 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:45.568137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d1c30-4d4a-492e-892d-a179c9e1a496 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:45.569004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d1c30-4d4a-492e-892d-a179c9e1a496 None admin] RBAC: Adding query filter params (role.id=45565d93c8c142e086e6b7ce62ff84e4, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=62bf4ef4fce74f7b91e453f73430d9ae) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:45.569313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d1c30-4d4a-492e-892d-a179c9e1a496 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:45.570778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d1c30-4d4a-492e-892d-a179c9e1a496 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:45.577438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 634/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:35:45 2022] GET /identity/v3/role_assignments?user.id=62bf4ef4fce74f7b91e453f73430d9ae&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=45565d93c8c142e086e6b7ce62ff84e4 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:35:47.832070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c530e9-f874-4bb4-b559-638795bc0d2f None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:47.832650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c530e9-f874-4bb4-b559-638795bc0d2f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:47.832650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c530e9-f874-4bb4-b559-638795bc0d2f None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:47.834210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 613/1247] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:35:47.839978 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bce3dc-4a2c-47d0-bba9-3ee0a4a1deae None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:47.840421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bce3dc-4a2c-47d0-bba9-3ee0a4a1deae None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:47.840525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bce3dc-4a2c-47d0-bba9-3ee0a4a1deae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:47.847619 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:47.860873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-24bce3dc-4a2c-47d0-bba9-3ee0a4a1deae None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:35:47.862172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-24bce3dc-4a2c-47d0-bba9-3ee0a4a1deae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:35:47.942541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 635/1248] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:35:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:35:47.967366 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c530e9-f874-4bb4-b559-638795bc0d2f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:47.986585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:48.089411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c530e9-f874-4bb4-b559-638795bc0d2f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:48.127918 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad27228-e75d-41e3-b3ca-35835559fb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:48.128756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad27228-e75d-41e3-b3ca-35835559fb0a None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:48.129039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad27228-e75d-41e3-b3ca-35835559fb0a None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:48.129307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad27228-e75d-41e3-b3ca-35835559fb0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:48.142286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad27228-e75d-41e3-b3ca-35835559fb0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-fad27228-e75d-41e3-b3ca-35835559fb0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:35:48.150937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:48.152093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:48.153083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:48.153083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:48.153083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:48.153083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:48.153083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:35:48.153083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:35:48.153083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:35:48.153083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:35:48.153083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:35:48.153083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:35:48.153083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:35:48.153083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:48.158063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 614/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:35:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:48.158883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bce3dc-4a2c-47d0-bba9-3ee0a4a1deae None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:48.273740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bce3dc-4a2c-47d0-bba9-3ee0a4a1deae None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:48.307744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:48.314037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b2381-77d5-4b0e-a485-bbebd186445c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:48.314717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b2381-77d5-4b0e-a485-bbebd186445c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:48.315008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b2381-77d5-4b0e-a485-bbebd186445c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:48.315295 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b2381-77d5-4b0e-a485-bbebd186445c None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:48.316080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b2381-77d5-4b0e-a485-bbebd186445c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:48.316391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b2381-77d5-4b0e-a485-bbebd186445c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:48.317824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b2381-77d5-4b0e-a485-bbebd186445c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:48.327641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 636/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:35:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:48.333882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad27228-e75d-41e3-b3ca-35835559fb0a None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:48.426606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:48.454418 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad27228-e75d-41e3-b3ca-35835559fb0a None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:48.492367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcacdf93-acd7-4b4e-902f-bac4b6b0f544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:48.493193 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacdf93-acd7-4b4e-902f-bac4b6b0f544 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:48.494135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacdf93-acd7-4b4e-902f-bac4b6b0f544 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:48.494401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacdf93-acd7-4b4e-902f-bac4b6b0f544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:48.507607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcacdf93-acd7-4b4e-902f-bac4b6b0f544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:48.509325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcacdf93-acd7-4b4e-902f-bac4b6b0f544 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-bcacdf93-acd7-4b4e-902f-bac4b6b0f544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:48.531736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:48.532917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:48.534516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:48.534516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:48.534516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:48.534516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:48.534516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 615/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:48.540064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b2381-77d5-4b0e-a485-bbebd186445c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:48.645994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b2381-77d5-4b0e-a485-bbebd186445c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:48.684755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da2cad-c132-47da-a5f0-0d27fe93759e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:48.685481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da2cad-c132-47da-a5f0-0d27fe93759e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:48.685794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da2cad-c132-47da-a5f0-0d27fe93759e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:48.686055 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da2cad-c132-47da-a5f0-0d27fe93759e None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:48.686778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da2cad-c132-47da-a5f0-0d27fe93759e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:48.687082 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da2cad-c132-47da-a5f0-0d27fe93759e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:48.688445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da2cad-c132-47da-a5f0-0d27fe93759e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:48.700808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 637/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:48.706689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcacdf93-acd7-4b4e-902f-bac4b6b0f544 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:48.815828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcacdf93-acd7-4b4e-902f-bac4b6b0f544 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:48.844349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:48.855187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d1c39-8484-4094-a1da-e6b07ed76fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:48.856176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d1c39-8484-4094-a1da-e6b07ed76fa8 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:48.856464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d1c39-8484-4094-a1da-e6b07ed76fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:48.856772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d1c39-8484-4094-a1da-e6b07ed76fa8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:48.866308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d1c39-8484-4094-a1da-e6b07ed76fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:48.867768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d1c39-8484-4094-a1da-e6b07ed76fa8 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-c25d1c39-8484-4094-a1da-e6b07ed76fa8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:35:48.882487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:48.883592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:48.885149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:48.885149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:48.885149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:35:48.885149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:35:48.885149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:35:48.885149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:35:48.885149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:35:48.885149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:35:48.885149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 08:35:48.885149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:48.885149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 616/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 08:35:48.890102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da2cad-c132-47da-a5f0-0d27fe93759e None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:48.947728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:48.994594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da2cad-c132-47da-a5f0-0d27fe93759e None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:49.031312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7e931-539e-4000-b45d-688bfb5142f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:49.031987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7e931-539e-4000-b45d-688bfb5142f4 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:49.032271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7e931-539e-4000-b45d-688bfb5142f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:49.032546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7e931-539e-4000-b45d-688bfb5142f4 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:49.033442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc7e931-539e-4000-b45d-688bfb5142f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:49.033726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc7e931-539e-4000-b45d-688bfb5142f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:49.035133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc7e931-539e-4000-b45d-688bfb5142f4 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:49.055347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 638/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Jul 25 08:35:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:49.061884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d1c39-8484-4094-a1da-e6b07ed76fa8 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:49.173586 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d1c39-8484-4094-a1da-e6b07ed76fa8 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:49.211382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48249f2e-058c-4c8a-a94d-f0c37bcea94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:49.211974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48249f2e-058c-4c8a-a94d-f0c37bcea94e None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:49.212290 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48249f2e-058c-4c8a-a94d-f0c37bcea94e None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:49.212559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48249f2e-058c-4c8a-a94d-f0c37bcea94e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:49.275105 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:49.289868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48249f2e-058c-4c8a-a94d-f0c37bcea94e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:49.292041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48249f2e-058c-4c8a-a94d-f0c37bcea94e None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:49.395154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 617/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:35:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:35:49.401902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7e931-539e-4000-b45d-688bfb5142f4 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:49.507658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7e931-539e-4000-b45d-688bfb5142f4 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:49.521352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:49.547832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bcc63-1835-4a11-bbf5-a69861b56162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:49.548663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bcc63-1835-4a11-bbf5-a69861b56162 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:49.548946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bcc63-1835-4a11-bbf5-a69861b56162 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:49.549277 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bcc63-1835-4a11-bbf5-a69861b56162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:49.559373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bcc63-1835-4a11-bbf5-a69861b56162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:49.560847 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bcc63-1835-4a11-bbf5-a69861b56162 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-346bcc63-1835-4a11-bbf5-a69861b56162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:49.578393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:49.579392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:49.580325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:49.580325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:49.580325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:49.580325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:49.580325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 639/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:49.585459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48249f2e-058c-4c8a-a94d-f0c37bcea94e None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:49.676956 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:49.691289 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48249f2e-058c-4c8a-a94d-f0c37bcea94e None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:49.728389 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648a7c1-f813-449d-aeae-61463a1da4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:49.729087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648a7c1-f813-449d-aeae-61463a1da4fc None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:49.729372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648a7c1-f813-449d-aeae-61463a1da4fc None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:49.729655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648a7c1-f813-449d-aeae-61463a1da4fc None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:49.730372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648a7c1-f813-449d-aeae-61463a1da4fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:49.730689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648a7c1-f813-449d-aeae-61463a1da4fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:49.732069 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648a7c1-f813-449d-aeae-61463a1da4fc None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:49.745263 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 618/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:49.749612 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bcc63-1835-4a11-bbf5-a69861b56162 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:49.866262 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bcc63-1835-4a11-bbf5-a69861b56162 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:49.908845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871cbbe-65af-49a1-b213-dc596c065019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:49.909649 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871cbbe-65af-49a1-b213-dc596c065019 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:49.909936 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871cbbe-65af-49a1-b213-dc596c065019 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:49.910215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871cbbe-65af-49a1-b213-dc596c065019 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:49.918756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871cbbe-65af-49a1-b213-dc596c065019 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:49.920230 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871cbbe-65af-49a1-b213-dc596c065019 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-5871cbbe-65af-49a1-b213-dc596c065019 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:35:49.934286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:49.935257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:49.936289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:35:49.936289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:35:49.936289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:35:49.936289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:35:49.936289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:35:49.936289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:35:49.936289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:35:49.936289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:49.936289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 640/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:35:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:49.941584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648a7c1-f813-449d-aeae-61463a1da4fc None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:50.043859 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648a7c1-f813-449d-aeae-61463a1da4fc None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:50.083701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054231d1-14bf-4a08-9595-e4d88b8b1358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:50.084370 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054231d1-14bf-4a08-9595-e4d88b8b1358 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:50.084688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054231d1-14bf-4a08-9595-e4d88b8b1358 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:50.084970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054231d1-14bf-4a08-9595-e4d88b8b1358 None admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:50.085762 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054231d1-14bf-4a08-9595-e4d88b8b1358 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:50.087139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054231d1-14bf-4a08-9595-e4d88b8b1358 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:50.087495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054231d1-14bf-4a08-9595-e4d88b8b1358 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:50.106718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 619/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:35:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:50.107946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871cbbe-65af-49a1-b213-dc596c065019 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:50.139344 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:50.224525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871cbbe-65af-49a1-b213-dc596c065019 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:50.261134 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e2ba5-9afc-4b4a-8c6e-5504541a7504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:50.261805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e2ba5-9afc-4b4a-8c6e-5504541a7504 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:50.262086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e2ba5-9afc-4b4a-8c6e-5504541a7504 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:50.262356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e2ba5-9afc-4b4a-8c6e-5504541a7504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:50.263194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e2ba5-9afc-4b4a-8c6e-5504541a7504 None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=62bf4ef4fce74f7b91e453f73430d9ae) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:50.263502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e2ba5-9afc-4b4a-8c6e-5504541a7504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:50.264979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e2ba5-9afc-4b4a-8c6e-5504541a7504 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:50.271365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 641/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:35:50 2022] GET /identity/v3/role_assignments?user.id=62bf4ef4fce74f7b91e453f73430d9ae&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:52.246002 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df757e-296f-4c89-903c-f36331f092ac None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:52.246533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df757e-296f-4c89-903c-f36331f092ac None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:52.246645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df757e-296f-4c89-903c-f36331f092ac None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:52.247941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 620/1261] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:35:52.253128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e0762-0f45-4f03-a132-86b152b4a324 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:52.253495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e0762-0f45-4f03-a132-86b152b4a324 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:52.253700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e0762-0f45-4f03-a132-86b152b4a324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:52.271734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d48e0762-0f45-4f03-a132-86b152b4a324 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:35:52.272976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d48e0762-0f45-4f03-a132-86b152b4a324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:35:52.338742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 642/1262] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:35:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:35:52.361354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df757e-296f-4c89-903c-f36331f092ac None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:52.371883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:52.459411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df757e-296f-4c89-903c-f36331f092ac None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:52.492367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e6300-beb3-4bd8-9e0b-5ffdc70b4ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:52.493204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e6300-beb3-4bd8-9e0b-5ffdc70b4ea1 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:52.493491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e6300-beb3-4bd8-9e0b-5ffdc70b4ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:52.493769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e6300-beb3-4bd8-9e0b-5ffdc70b4ea1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:52.503609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e6300-beb3-4bd8-9e0b-5ffdc70b4ea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-813e6300-beb3-4bd8-9e0b-5ffdc70b4ea1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:35:52.511130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:52.512211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:52.513355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 621/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:35:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:52.518136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e0762-0f45-4f03-a132-86b152b4a324 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:52.536476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:52.616496 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e0762-0f45-4f03-a132-86b152b4a324 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:52.651374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d40b8-1b30-473d-868a-37014a047b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:52.652039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d40b8-1b30-473d-868a-37014a047b89 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:52.652327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d40b8-1b30-473d-868a-37014a047b89 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:52.652631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d40b8-1b30-473d-868a-37014a047b89 None admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:52.653380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d40b8-1b30-473d-868a-37014a047b89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:52.653686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d40b8-1b30-473d-868a-37014a047b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:52.655041 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d40b8-1b30-473d-868a-37014a047b89 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:52.663000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 643/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:35:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:52.668617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e6300-beb3-4bd8-9e0b-5ffdc70b4ea1 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:52.731875 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:52.763894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e6300-beb3-4bd8-9e0b-5ffdc70b4ea1 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:52.795897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af019299-b8f0-47eb-800b-972dd618d9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:52.796696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af019299-b8f0-47eb-800b-972dd618d9aa None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:52.796981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af019299-b8f0-47eb-800b-972dd618d9aa None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:52.797257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af019299-b8f0-47eb-800b-972dd618d9aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:52.807280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af019299-b8f0-47eb-800b-972dd618d9aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:52.808750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af019299-b8f0-47eb-800b-972dd618d9aa None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-af019299-b8f0-47eb-800b-972dd618d9aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:52.825580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:52.826617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:52.827728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:52.827728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:52.827728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:52.827728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:52.827728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 622/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:52.832213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d40b8-1b30-473d-868a-37014a047b89 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:52.911157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:52.929579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d40b8-1b30-473d-868a-37014a047b89 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:52.961484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a8f47-ab5d-40b0-a9e7-fabb3565bdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:52.962711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a8f47-ab5d-40b0-a9e7-fabb3565bdd1 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:52.962990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a8f47-ab5d-40b0-a9e7-fabb3565bdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:52.963261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a8f47-ab5d-40b0-a9e7-fabb3565bdd1 None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:52.963971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a8f47-ab5d-40b0-a9e7-fabb3565bdd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:52.964273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a8f47-ab5d-40b0-a9e7-fabb3565bdd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:52.965640 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a8f47-ab5d-40b0-a9e7-fabb3565bdd1 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:52.976890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 644/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:52.982357 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af019299-b8f0-47eb-800b-972dd618d9aa None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:53.081318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af019299-b8f0-47eb-800b-972dd618d9aa None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:53.102460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:53.118759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f253e8-5b54-4dfa-9cc6-3dfa46a95c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:53.119561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f253e8-5b54-4dfa-9cc6-3dfa46a95c4b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:53.119845 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f253e8-5b54-4dfa-9cc6-3dfa46a95c4b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:53.120124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f253e8-5b54-4dfa-9cc6-3dfa46a95c4b None admin] PATH_INFO: `/v3/users/placement` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:53.128289 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f253e8-5b54-4dfa-9cc6-3dfa46a95c4b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:53.129784 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f253e8-5b54-4dfa-9cc6-3dfa46a95c4b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-f8f253e8-5b54-4dfa-9cc6-3dfa46a95c4b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:35:53.144373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:53.145307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:53.146731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:53.146731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:53.146731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:35:53.146731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:35:53.146731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:35:53.146731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:35:53.146731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:35:53.146731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:35:53.146731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 08:35:53.146731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:53.146731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 623/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:53.151700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a8f47-ab5d-40b0-a9e7-fabb3565bdd1 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:53.246364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a8f47-ab5d-40b0-a9e7-fabb3565bdd1 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:53.279123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e350e-57a9-47de-80c6-909927346902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:53.279775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e350e-57a9-47de-80c6-909927346902 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:53.280057 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e350e-57a9-47de-80c6-909927346902 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:53.280330 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e350e-57a9-47de-80c6-909927346902 None admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:53.281187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e350e-57a9-47de-80c6-909927346902 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:53.281492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e350e-57a9-47de-80c6-909927346902 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:53.282899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e350e-57a9-47de-80c6-909927346902 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:53.291300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:53.299994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 645/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Jul 25 08:35:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:53.306116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f253e8-5b54-4dfa-9cc6-3dfa46a95c4b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:53.402717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f253e8-5b54-4dfa-9cc6-3dfa46a95c4b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:53.435404 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235a0e0-04ee-42b0-b20a-8b52022cc6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:53.436179 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235a0e0-04ee-42b0-b20a-8b52022cc6fa None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:53.436464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235a0e0-04ee-42b0-b20a-8b52022cc6fa None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:53.436773 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235a0e0-04ee-42b0-b20a-8b52022cc6fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:53.446937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235a0e0-04ee-42b0-b20a-8b52022cc6fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:53.448376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235a0e0-04ee-42b0-b20a-8b52022cc6fa None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-b235a0e0-04ee-42b0-b20a-8b52022cc6fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:53.464896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:53.465905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:53.466973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:53.466973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:53.466973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:53.466973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:53.466973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 624/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:53.471515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e350e-57a9-47de-80c6-909927346902 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:53.568535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e350e-57a9-47de-80c6-909927346902 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:53.600898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9159e-b4e9-4fdb-a056-9d395e58347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:53.601554 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9159e-b4e9-4fdb-a056-9d395e58347c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:53.601841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9159e-b4e9-4fdb-a056-9d395e58347c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:53.602112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9159e-b4e9-4fdb-a056-9d395e58347c None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:53.603317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9159e-b4e9-4fdb-a056-9d395e58347c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:53.603618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9159e-b4e9-4fdb-a056-9d395e58347c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:53.604986 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9159e-b4e9-4fdb-a056-9d395e58347c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:53.616014 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 646/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:35:53.621798 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235a0e0-04ee-42b0-b20a-8b52022cc6fa None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:53.652433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:53.717909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235a0e0-04ee-42b0-b20a-8b52022cc6fa None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:53.752484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f61495-f485-4a05-aa06-5a02e33045b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:53.753286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f61495-f485-4a05-aa06-5a02e33045b6 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:53.753572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f61495-f485-4a05-aa06-5a02e33045b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:53.753847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f61495-f485-4a05-aa06-5a02e33045b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:53.760342 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f61495-f485-4a05-aa06-5a02e33045b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:53.761809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f61495-f485-4a05-aa06-5a02e33045b6 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-e4f61495-f485-4a05-aa06-5a02e33045b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:35:53.776497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:53.777513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:53.778739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:35:53.778739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:35:53.778739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:35:53.778739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:35:53.778739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:35:53.778739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:35:53.778739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:35:53.778739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:53.778739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 625/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:35:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:53.783279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9159e-b4e9-4fdb-a056-9d395e58347c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:53.840223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:53.881464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9159e-b4e9-4fdb-a056-9d395e58347c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:53.914029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a979e3-66f1-4f66-8515-6a1e5d425569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:53.914682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a979e3-66f1-4f66-8515-6a1e5d425569 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:53.915571 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a979e3-66f1-4f66-8515-6a1e5d425569 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:53.916192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a979e3-66f1-4f66-8515-6a1e5d425569 None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:53.917011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a979e3-66f1-4f66-8515-6a1e5d425569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:53.917320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a979e3-66f1-4f66-8515-6a1e5d425569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:53.918751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a979e3-66f1-4f66-8515-6a1e5d425569 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:53.930572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 647/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:35:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:53.936407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f61495-f485-4a05-aa06-5a02e33045b6 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:54.023314 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:54.032412 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f61495-f485-4a05-aa06-5a02e33045b6 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:54.066719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660fd3d-fc9f-4474-aa42-8f06671be817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:54.067621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660fd3d-fc9f-4474-aa42-8f06671be817 None admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:54.067913 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660fd3d-fc9f-4474-aa42-8f06671be817 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:54.068219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660fd3d-fc9f-4474-aa42-8f06671be817 None admin] PATH_INFO: `/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/62bf4ef4fce74f7b91e453f73430d9ae/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:54.070271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660fd3d-fc9f-4474-aa42-8f06671be817 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ee27c4da77c47428e02fd65e890d256, user_id=62bf4ef4fce74f7b91e453f73430d9ae, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:54.071689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660fd3d-fc9f-4474-aa42-8f06671be817 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:54.084474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 626/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Jul 25 08:35:53 2022] PUT /identity/v3/projects/2ee27c4da77c47428e02fd65e890d256/users/62bf4ef4fce74f7b91e453f73430d9ae/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:35:56.010230 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52604c-9f97-4a35-a943-1cc232d56818 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:56.010785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52604c-9f97-4a35-a943-1cc232d56818 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:56.010785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52604c-9f97-4a35-a943-1cc232d56818 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:56.012184 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 648/1274] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:35:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:35:56.017338 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acff41-2279-46d7-ba94-feafc1da3aef None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:56.017713 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acff41-2279-46d7-ba94-feafc1da3aef None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:56.017897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acff41-2279-46d7-ba94-feafc1da3aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:56.035746 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-83acff41-2279-46d7-ba94-feafc1da3aef None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:35:56.037006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-83acff41-2279-46d7-ba94-feafc1da3aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:35:56.108268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 627/1275] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:35:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:35:56.131469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52604c-9f97-4a35-a943-1cc232d56818 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:56.230885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52604c-9f97-4a35-a943-1cc232d56818 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:56.247794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:56.264875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f064dee-d64a-4d5d-881d-06a2caa764f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:56.265600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f064dee-d64a-4d5d-881d-06a2caa764f0 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:56.265886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f064dee-d64a-4d5d-881d-06a2caa764f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:56.266163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f064dee-d64a-4d5d-881d-06a2caa764f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:56.276280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f064dee-d64a-4d5d-881d-06a2caa764f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-2f064dee-d64a-4d5d-881d-06a2caa764f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 08:35:56.283902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:56.284999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:56.285910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 649/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Jul 25 08:35:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:56.291086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acff41-2279-46d7-ba94-feafc1da3aef None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:56.370237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:56.389520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acff41-2279-46d7-ba94-feafc1da3aef None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:56.422954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd644-b984-4d13-ada7-4c81706527a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:56.423611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd644-b984-4d13-ada7-4c81706527a6 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:56.423890 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd644-b984-4d13-ada7-4c81706527a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:56.424161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd644-b984-4d13-ada7-4c81706527a6 None admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:56.424937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cd644-b984-4d13-ada7-4c81706527a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:56.425244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cd644-b984-4d13-ada7-4c81706527a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:56.426603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cd644-b984-4d13-ada7-4c81706527a6 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:56.434950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 628/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Jul 25 08:35:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:56.440600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f064dee-d64a-4d5d-881d-06a2caa764f0 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:56.536712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f064dee-d64a-4d5d-881d-06a2caa764f0 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:56.569369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8cc10-9382-4d5a-ac88-5e0c2c16548c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:56.570137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8cc10-9382-4d5a-ac88-5e0c2c16548c None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:56.570421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8cc10-9382-4d5a-ac88-5e0c2c16548c None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:56.570699 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8cc10-9382-4d5a-ac88-5e0c2c16548c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:56.580958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8cc10-9382-4d5a-ac88-5e0c2c16548c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:56.582381 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8cc10-9382-4d5a-ac88-5e0c2c16548c None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-4cb8cc10-9382-4d5a-ac88-5e0c2c16548c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:56.598940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:56.600008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:56.601298 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:56.601298 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:56.601298 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:56.601298 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:56.601298 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 650/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:56.606048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd644-b984-4d13-ada7-4c81706527a6 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:56.701264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd644-b984-4d13-ada7-4c81706527a6 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:56.733014 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77ba3d-4429-451c-bc31-0cb0c1f8ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:56.733655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77ba3d-4429-451c-bc31-0cb0c1f8ade2 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:56.733937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77ba3d-4429-451c-bc31-0cb0c1f8ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:56.734210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77ba3d-4429-451c-bc31-0cb0c1f8ade2 None admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:56.734923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77ba3d-4429-451c-bc31-0cb0c1f8ade2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:56.735225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77ba3d-4429-451c-bc31-0cb0c1f8ade2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:56.736564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77ba3d-4429-451c-bc31-0cb0c1f8ade2 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:56.748502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 629/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:56.753794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8cc10-9382-4d5a-ac88-5e0c2c16548c None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:56.784133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:56.851721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8cc10-9382-4d5a-ac88-5e0c2c16548c None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:56.884126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f19999-ab8c-4fd3-9220-1ba86615d136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:56.884935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f19999-ab8c-4fd3-9220-1ba86615d136 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:56.885220 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f19999-ab8c-4fd3-9220-1ba86615d136 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:56.885493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f19999-ab8c-4fd3-9220-1ba86615d136 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:56.892724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f19999-ab8c-4fd3-9220-1ba86615d136 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:56.894143 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f19999-ab8c-4fd3-9220-1ba86615d136 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-98f19999-ab8c-4fd3-9220-1ba86615d136 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 08:35:56.908910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:56.909988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:56.911281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:56.911281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:56.911281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 08:35:56.911281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 08:35:56.911281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 08:35:56.911281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 08:35:56.911281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 08:35:56.911281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 08:35:56.911281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 08:35:56.911281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:35:56.911281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 651/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:56.916237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77ba3d-4429-451c-bc31-0cb0c1f8ade2 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:56.920226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:57.011731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77ba3d-4429-451c-bc31-0cb0c1f8ade2 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:57.045044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025f50b-5dd6-45b4-b104-7ca09832184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:57.045851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025f50b-5dd6-45b4-b104-7ca09832184b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:57.046163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025f50b-5dd6-45b4-b104-7ca09832184b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:57.046510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025f50b-5dd6-45b4-b104-7ca09832184b None admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:57.047676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025f50b-5dd6-45b4-b104-7ca09832184b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:57.048025 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025f50b-5dd6-45b4-b104-7ca09832184b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:57.049885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025f50b-5dd6-45b4-b104-7ca09832184b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:57.069243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 630/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Jul 25 08:35:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:57.074830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f19999-ab8c-4fd3-9220-1ba86615d136 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:57.165236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:57.175047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f19999-ab8c-4fd3-9220-1ba86615d136 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:57.207414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e1b8a-72eb-4521-a807-9f8137a708b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:57.208052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e1b8a-72eb-4521-a807-9f8137a708b1 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:57.208338 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e1b8a-72eb-4521-a807-9f8137a708b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:57.208641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e1b8a-72eb-4521-a807-9f8137a708b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:57.273954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e1b8a-72eb-4521-a807-9f8137a708b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:57.275946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e1b8a-72eb-4521-a807-9f8137a708b1 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:57.363860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:57.374473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 652/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Jul 25 08:35:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 08:35:57.380215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025f50b-5dd6-45b4-b104-7ca09832184b None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:57.439107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:57.476382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025f50b-5dd6-45b4-b104-7ca09832184b None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:57.509138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328ad07-4eac-42ab-b5d4-9f092dea13a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:57.509916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328ad07-4eac-42ab-b5d4-9f092dea13a8 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:57.510199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328ad07-4eac-42ab-b5d4-9f092dea13a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:57.510474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328ad07-4eac-42ab-b5d4-9f092dea13a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:57.520049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328ad07-4eac-42ab-b5d4-9f092dea13a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:57.521785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328ad07-4eac-42ab-b5d4-9f092dea13a8 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-9328ad07-4eac-42ab-b5d4-9f092dea13a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 08:35:57.538485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:57.539574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:57.540735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 08:35:57.540735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 08:35:57.540735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 08:35:57.540735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:57.540735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 631/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:57.545201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e1b8a-72eb-4521-a807-9f8137a708b1 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:57.641973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e1b8a-72eb-4521-a807-9f8137a708b1 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:57.674647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e3353-4dea-4c35-be68-4f874db091cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:57.675304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3353-4dea-4c35-be68-4f874db091cc None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:57.675591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3353-4dea-4c35-be68-4f874db091cc None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:57.675871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3353-4dea-4c35-be68-4f874db091cc None admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:57.676607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e3353-4dea-4c35-be68-4f874db091cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:57.676918 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e3353-4dea-4c35-be68-4f874db091cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:57.678277 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e3353-4dea-4c35-be68-4f874db091cc None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:57.689823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 653/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Jul 25 08:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:35:57.695364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328ad07-4eac-42ab-b5d4-9f092dea13a8 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:57.791082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328ad07-4eac-42ab-b5d4-9f092dea13a8 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:57.804398 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:57.824722 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb44d3-7e89-4d1c-a5da-f0c4a7ea7553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:57.825494 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb44d3-7e89-4d1c-a5da-f0c4a7ea7553 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:57.825775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb44d3-7e89-4d1c-a5da-f0c4a7ea7553 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:57.826049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb44d3-7e89-4d1c-a5da-f0c4a7ea7553 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:57.832440 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb44d3-7e89-4d1c-a5da-f0c4a7ea7553 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:57.833906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb44d3-7e89-4d1c-a5da-f0c4a7ea7553 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application [None req-11bb44d3-7e89-4d1c-a5da-f0c4a7ea7553 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 08:35:57.847184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application with Lock( Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:35:57.848167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:35:57.849407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 08:35:57.849407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 08:35:57.849407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 08:35:57.849407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 08:35:57.849407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 08:35:57.849407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 08:35:57.849407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 08:35:57.849407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: ERROR keystone.server.flask.application Jul 25 08:35:57.849407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 632/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Jul 25 08:35:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:35:57.854150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e3353-4dea-4c35-be68-4f874db091cc None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:57.904085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:35:57.954008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e3353-4dea-4c35-be68-4f874db091cc None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:57.986349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e650ab7-85dd-4279-9054-656d15e1590e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:57.987011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e650ab7-85dd-4279-9054-656d15e1590e None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:57.987298 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e650ab7-85dd-4279-9054-656d15e1590e None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:57.987579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e650ab7-85dd-4279-9054-656d15e1590e None admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:57.988364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e650ab7-85dd-4279-9054-656d15e1590e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:57.988706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e650ab7-85dd-4279-9054-656d15e1590e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:57.990136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e650ab7-85dd-4279-9054-656d15e1590e None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:58.002427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 654/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Jul 25 08:35:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:35:58.008392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb44d3-7e89-4d1c-a5da-f0c4a7ea7553 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:35:58.113374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb44d3-7e89-4d1c-a5da-f0c4a7ea7553 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:35:58.146112 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ee5c3-51ac-412f-b28b-6bd0eed5b43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:35:58.146770 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ee5c3-51ac-412f-b28b-6bd0eed5b43b None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:35:58.147057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ee5c3-51ac-412f-b28b-6bd0eed5b43b None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:35:58.147336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ee5c3-51ac-412f-b28b-6bd0eed5b43b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:35:58.148174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ee5c3-51ac-412f-b28b-6bd0eed5b43b None admin] RBAC: Adding query filter params (role.id=996428c7fd054e5cb6a1aaa31f27cf84, scope.project.id=2ee27c4da77c47428e02fd65e890d256, user.id=62bf4ef4fce74f7b91e453f73430d9ae) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:35:58.148488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ee5c3-51ac-412f-b28b-6bd0eed5b43b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:35:58.149995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ee5c3-51ac-412f-b28b-6bd0eed5b43b None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:35:58.155712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 633/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Jul 25 08:35:58 2022] GET /identity/v3/role_assignments?user.id=62bf4ef4fce74f7b91e453f73430d9ae&scope.project.id=2ee27c4da77c47428e02fd65e890d256&role.id=996428c7fd054e5cb6a1aaa31f27cf84 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:36:00.137414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7a32f-79d4-4883-9692-080289c72e05 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:00.137943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7a32f-79d4-4883-9692-080289c72e05 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:00.137943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7a32f-79d4-4883-9692-080289c72e05 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:00.139363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 655/1288] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:36:00.144490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ea804-a3dd-491c-9fec-a9e14bbf7990 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:00.144941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ea804-a3dd-491c-9fec-a9e14bbf7990 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:00.145083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ea804-a3dd-491c-9fec-a9e14bbf7990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:00.163072 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-a99ea804-a3dd-491c-9fec-a9e14bbf7990 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:36:00.164240 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a99ea804-a3dd-491c-9fec-a9e14bbf7990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:36:00.193171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:36:00.229189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 634/1289] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:36:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:36:00.251049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7a32f-79d4-4883-9692-080289c72e05 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:36:00.350701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7a32f-79d4-4883-9692-080289c72e05 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:36:00.361490 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:36:00.385390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f408786-ac0e-4e67-a4d0-32b312c20bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:36:00.386185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f408786-ac0e-4e67-a4d0-32b312c20bcb None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:00.386473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f408786-ac0e-4e67-a4d0-32b312c20bcb None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:00.386754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f408786-ac0e-4e67-a4d0-32b312c20bcb None admin] PATH_INFO: `/v3/services/placement` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:00.387544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f408786-ac0e-4e67-a4d0-32b312c20bcb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:36:00.388881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f408786-ac0e-4e67-a4d0-32b312c20bcb None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-5f408786-ac0e-4e67-a4d0-32b312c20bcb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:36:00.400932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:36:00.402122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:36:00.403359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 656/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Jul 25 08:36:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:36:00.408127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ea804-a3dd-491c-9fec-a9e14bbf7990 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:36:00.502361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ea804-a3dd-491c-9fec-a9e14bbf7990 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:36:00.534901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92299315-a986-47ec-b02f-a28af981473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:36:00.535561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92299315-a986-47ec-b02f-a28af981473f None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:00.535846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92299315-a986-47ec-b02f-a28af981473f None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:00.536141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92299315-a986-47ec-b02f-a28af981473f None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:00.536921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92299315-a986-47ec-b02f-a28af981473f None admin] RBAC: Adding query filter params (name=placement) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:36:00.537231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92299315-a986-47ec-b02f-a28af981473f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:36:00.538592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92299315-a986-47ec-b02f-a28af981473f None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:36:00.543898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 635/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Mon Jul 25 08:36:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:36:00.549499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f408786-ac0e-4e67-a4d0-32b312c20bcb None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:36:00.646587 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f408786-ac0e-4e67-a4d0-32b312c20bcb None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:36:00.679955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3b1a9-695f-4235-83d6-36d9084dd9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:36:00.680656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3b1a9-695f-4235-83d6-36d9084dd9cc None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:00.680947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3b1a9-695f-4235-83d6-36d9084dd9cc None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:00.681229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3b1a9-695f-4235-83d6-36d9084dd9cc None admin] PATH_INFO: `/v3/services` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:00.681937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3b1a9-695f-4235-83d6-36d9084dd9cc None admin] RBAC: Adding query filter params (type=placement) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:36:00.682247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3b1a9-695f-4235-83d6-36d9084dd9cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:36:00.683607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3b1a9-695f-4235-83d6-36d9084dd9cc None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:36:00.687937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 657/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Mon Jul 25 08:36:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:36:02.634076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca949f75-133b-44f4-b0c9-ca7b6b0b1da2 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:02.634556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca949f75-133b-44f4-b0c9-ca7b6b0b1da2 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:02.634556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca949f75-133b-44f4-b0c9-ca7b6b0b1da2 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:02.635931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 636/1293] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:36:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:36:02.641116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe27d6-a285-457c-8f79-7756c0fffe6e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:02.641504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe27d6-a285-457c-8f79-7756c0fffe6e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:02.641680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe27d6-a285-457c-8f79-7756c0fffe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:02.660659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-e7fe27d6-a285-457c-8f79-7756c0fffe6e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:36:02.661861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e7fe27d6-a285-457c-8f79-7756c0fffe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:36:02.690507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:36:02.726951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 658/1294] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:36:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:36:02.749166 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca949f75-133b-44f4-b0c9-ca7b6b0b1da2 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:36:02.779733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:36:02.848501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca949f75-133b-44f4-b0c9-ca7b6b0b1da2 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:36:02.881270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b20e7-e8fa-4a50-98cb-5dd4c062f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:36:02.881928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b20e7-e8fa-4a50-98cb-5dd4c062f0ab None admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:02.882213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b20e7-e8fa-4a50-98cb-5dd4c062f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:02.882491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b20e7-e8fa-4a50-98cb-5dd4c062f0ab None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:02.883803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b20e7-e8fa-4a50-98cb-5dd4c062f0ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:36:02.885189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b20e7-e8fa-4a50-98cb-5dd4c062f0ab None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:36:02.898932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 637/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Jul 25 08:36:02 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:36:04.855528 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50e160-ae9f-4dcb-ae54-aa0dae14a5b4 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:04.856113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50e160-ae9f-4dcb-ae54-aa0dae14a5b4 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:04.856113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50e160-ae9f-4dcb-ae54-aa0dae14a5b4 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:04.857618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 659/1296] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:36:04.862822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae0080-b8f8-424c-9be2-6cb9f99c9340 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:04.863238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae0080-b8f8-424c-9be2-6cb9f99c9340 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:04.863389 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae0080-b8f8-424c-9be2-6cb9f99c9340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:04.881078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-a8ae0080-b8f8-424c-9be2-6cb9f99c9340 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:36:04.882267 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a8ae0080-b8f8-424c-9be2-6cb9f99c9340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:36:04.953230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 638/1297] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:36:04 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:36:04.975765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50e160-ae9f-4dcb-ae54-aa0dae14a5b4 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:36:05.083010 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50e160-ae9f-4dcb-ae54-aa0dae14a5b4 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:36:05.118010 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d4556-108f-423b-949b-2276e73b17d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:36:05.118712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d4556-108f-423b-949b-2276e73b17d7 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:05.119042 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d4556-108f-423b-949b-2276e73b17d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:05.119380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d4556-108f-423b-949b-2276e73b17d7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:05.120082 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d4556-108f-423b-949b-2276e73b17d7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:36:05.121620 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d4556-108f-423b-949b-2276e73b17d7 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-ec5d4556-108f-423b-949b-2276e73b17d7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:36:05.134626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:36:05.135901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:36:05.137047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 660/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Jul 25 08:36:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:36:05.142035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae0080-b8f8-424c-9be2-6cb9f99c9340 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:36:05.160388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:36:05.239680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae0080-b8f8-424c-9be2-6cb9f99c9340 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:36:05.273165 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963d38b-bafe-48db-8302-5fab9241b2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:36:05.273823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963d38b-bafe-48db-8302-5fab9241b2a7 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:05.274104 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963d38b-bafe-48db-8302-5fab9241b2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:05.274378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963d38b-bafe-48db-8302-5fab9241b2a7 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:05.275081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963d38b-bafe-48db-8302-5fab9241b2a7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:36:05.275385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963d38b-bafe-48db-8302-5fab9241b2a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:36:05.276769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963d38b-bafe-48db-8302-5fab9241b2a7 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:36:05.281756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 639/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Mon Jul 25 08:36:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:36:05.287478 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d4556-108f-423b-949b-2276e73b17d7 None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:36:05.318188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:36:05.385947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d4556-108f-423b-949b-2276e73b17d7 None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:36:05.418232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda54d2-3dea-4c4c-91e9-c4e80ede2192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:36:05.418873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda54d2-3dea-4c4c-91e9-c4e80ede2192 None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:05.419154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda54d2-3dea-4c4c-91e9-c4e80ede2192 None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:05.419431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda54d2-3dea-4c4c-91e9-c4e80ede2192 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:05.420210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda54d2-3dea-4c4c-91e9-c4e80ede2192 None admin] RBAC: Adding query filter params (interface=public, service_id=9e747c2e42a24a3f97368c698516d55b, region_id=RegionOne) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:36:05.420519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda54d2-3dea-4c4c-91e9-c4e80ede2192 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:36:05.421894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda54d2-3dea-4c4c-91e9-c4e80ede2192 None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:36:05.426792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 661/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Jul 25 08:36:05 2022] GET /identity/v3/endpoints?service_id=9e747c2e42a24a3f97368c698516d55b&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:36:05.433137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963d38b-bafe-48db-8302-5fab9241b2a7 None admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:36:05.511672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:36:05.529377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963d38b-bafe-48db-8302-5fab9241b2a7 None admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:36:05.562060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef079f1-48c0-4e78-b26f-66580990d2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:36:05.562711 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef079f1-48c0-4e78-b26f-66580990d2a8 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:05.562993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef079f1-48c0-4e78-b26f-66580990d2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:05.563269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef079f1-48c0-4e78-b26f-66580990d2a8 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:05.563941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef079f1-48c0-4e78-b26f-66580990d2a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:36:05.565318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef079f1-48c0-4e78-b26f-66580990d2a8 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:36:05.571960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 640/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Jul 25 08:36:05 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:36:07.504165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f888054-611a-41f0-885a-8454769f2a7d None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:07.504738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f888054-611a-41f0-885a-8454769f2a7d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:07.504738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f888054-611a-41f0-885a-8454769f2a7d None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:07.506143 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 662/1302] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:36:07.511235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16801ba9-7e89-4917-ab66-097f5d8f46f2 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:07.511664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16801ba9-7e89-4917-ab66-097f5d8f46f2 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:07.511799 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16801ba9-7e89-4917-ab66-097f5d8f46f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:07.530167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-16801ba9-7e89-4917-ab66-097f5d8f46f2 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:36:07.531347 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-16801ba9-7e89-4917-ab66-097f5d8f46f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:36:07.595689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 641/1303] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:36:07 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 08:36:07.618076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f888054-611a-41f0-885a-8454769f2a7d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:36:07.710698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:36:07.718121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f888054-611a-41f0-885a-8454769f2a7d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:36:07.750128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da646783-0f24-4b16-808b-0365d84bb7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:36:07.750920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da646783-0f24-4b16-808b-0365d84bb7aa None admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:07.751203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da646783-0f24-4b16-808b-0365d84bb7aa None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:07.751479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da646783-0f24-4b16-808b-0365d84bb7aa None admin] PATH_INFO: `/v3/services/placement` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:07.752123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da646783-0f24-4b16-808b-0365d84bb7aa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:36:07.753498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da646783-0f24-4b16-808b-0365d84bb7aa None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application [None req-da646783-0f24-4b16-808b-0365d84bb7aa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 08:36:07.765807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 08:36:07.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application with Lock( Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self._enter() Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application return self.creator() Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application created_value = creator( Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: ERROR keystone.server.flask.application Jul 25 08:36:07.767647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 663/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Jul 25 08:36:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 08:36:07.773194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16801ba9-7e89-4917-ab66-097f5d8f46f2 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:36:07.848365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:36:07.871120 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16801ba9-7e89-4917-ab66-097f5d8f46f2 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:36:07.905633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa53d02-2d4d-4ac4-97bd-5317abae6af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:36:07.906769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa53d02-2d4d-4ac4-97bd-5317abae6af4 None admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:07.907287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa53d02-2d4d-4ac4-97bd-5317abae6af4 None admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:07.907811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa53d02-2d4d-4ac4-97bd-5317abae6af4 None admin] PATH_INFO: `/v3/services` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:07.909214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa53d02-2d4d-4ac4-97bd-5317abae6af4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:36:07.909793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa53d02-2d4d-4ac4-97bd-5317abae6af4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:36:07.912397 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa53d02-2d4d-4ac4-97bd-5317abae6af4 None admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:36:07.918415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 642/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Mon Jul 25 08:36:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:36:07.924245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da646783-0f24-4b16-808b-0365d84bb7aa None admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:36:08.023242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da646783-0f24-4b16-808b-0365d84bb7aa None admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:36:08.058383 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedb938-b8ec-4393-8319-15608043c4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:36:08.059065 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedb938-b8ec-4393-8319-15608043c4fd None admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:36:08.059432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedb938-b8ec-4393-8319-15608043c4fd None admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:36:08.059715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedb938-b8ec-4393-8319-15608043c4fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:36:08.060464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedb938-b8ec-4393-8319-15608043c4fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:36:08.061971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedb938-b8ec-4393-8319-15608043c4fd None admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:36:08.082739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 664/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:36:07 2022] POST /identity/v3/endpoints => generated 331 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:36:17.915446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=30 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:36:18.078690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:10.622660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84174dc8-81c4-4495-816d-49dca57afef3 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:10.623649 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84174dc8-81c4-4495-816d-49dca57afef3 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:10.624143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84174dc8-81c4-4495-816d-49dca57afef3 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:10.630330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 643/1307] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:10.631816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7edbf6-c977-4028-acb7-64a1f6c8b853 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:10.632244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7edbf6-c977-4028-acb7-64a1f6c8b853 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:10.632514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7edbf6-c977-4028-acb7-64a1f6c8b853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:10.760635 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4b7edbf6-c977-4028-acb7-64a1f6c8b853 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:10.761877 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4b7edbf6-c977-4028-acb7-64a1f6c8b853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:10.879613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 665/1308] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:10.934085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1b0a3-01d4-4668-b3a4-727bc7d60aa8 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:10.934385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1b0a3-01d4-4668-b3a4-727bc7d60aa8 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:10.934668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1b0a3-01d4-4668-b3a4-727bc7d60aa8 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:10.936059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 644/1309] 10.222.0.52 () {54 vars in 965 bytes} [Mon Jul 25 08:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:10.942196 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02329197-1c49-4232-8ce0-74f4c47dbdb1 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:10.942493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02329197-1c49-4232-8ce0-74f4c47dbdb1 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:10.942780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02329197-1c49-4232-8ce0-74f4c47dbdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:11.043710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-02329197-1c49-4232-8ce0-74f4c47dbdb1 None None] MFA Rules not processed for user `62688bc3d8a449c5bc9e0d64cd2462b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:11.045011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-02329197-1c49-4232-8ce0-74f4c47dbdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:11.067438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:11.125943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 666/1310] 10.222.0.52 () {58 vars in 1062 bytes} [Mon Jul 25 08:37:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:37:11.134804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1b0a3-01d4-4668-b3a4-727bc7d60aa8 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:11.136871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ada1b0a3-01d4-4668-b3a4-727bc7d60aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:11.246278 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1b0a3-01d4-4668-b3a4-727bc7d60aa8 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:11.329565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d98081-3970-41d5-ac54-228f20d7e7be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:11.330221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d98081-3970-41d5-ac54-228f20d7e7be service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:11.330505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d98081-3970-41d5-ac54-228f20d7e7be service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:11.330780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d98081-3970-41d5-ac54-228f20d7e7be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:11.332960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-22d98081-3970-41d5-ac54-228f20d7e7be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:11.389771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:11.449683 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d98081-3970-41d5-ac54-228f20d7e7be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:11.451126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d98081-3970-41d5-ac54-228f20d7e7be service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:11.470854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 645/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:37:13.762621 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d06435-8efe-4b41-8ad9-5c53ca210a69 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:13.763224 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d06435-8efe-4b41-8ad9-5c53ca210a69 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:13.763224 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d06435-8efe-4b41-8ad9-5c53ca210a69 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:13.764603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 667/1312] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:13.769928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec1e0a-f9cf-48ac-9ea9-7527397be32c None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:13.770396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec1e0a-f9cf-48ac-9ea9-7527397be32c None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:13.770494 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec1e0a-f9cf-48ac-9ea9-7527397be32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:13.843801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ecec1e0a-f9cf-48ac-9ea9-7527397be32c None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:13.845055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ecec1e0a-f9cf-48ac-9ea9-7527397be32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:13.951348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 646/1313] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:14.006862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b518439-152d-49be-8e71-6ef452814ebe None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:14.007151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b518439-152d-49be-8e71-6ef452814ebe None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:14.007436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b518439-152d-49be-8e71-6ef452814ebe None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:14.009251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 668/1314] 10.222.0.52 () {54 vars in 965 bytes} [Mon Jul 25 08:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:14.015703 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf93cea-b071-453f-8384-644e37b219c1 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:14.016339 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf93cea-b071-453f-8384-644e37b219c1 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:14.016660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf93cea-b071-453f-8384-644e37b219c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:14.030162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:14.095902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-dbf93cea-b071-453f-8384-644e37b219c1 None None] MFA Rules not processed for user `62688bc3d8a449c5bc9e0d64cd2462b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:14.097142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-dbf93cea-b071-453f-8384-644e37b219c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:14.174777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 647/1315] 10.222.0.52 () {58 vars in 1062 bytes} [Mon Jul 25 08:37:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:14.184317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b518439-152d-49be-8e71-6ef452814ebe None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:14.186034 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4b518439-152d-49be-8e71-6ef452814ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:14.315071 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b518439-152d-49be-8e71-6ef452814ebe None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:14.387656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:14.390480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdbd87-4c57-4a1d-b548-db875aaf13e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:14.391339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdbd87-4c57-4a1d-b548-db875aaf13e7 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:14.391672 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdbd87-4c57-4a1d-b548-db875aaf13e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:14.392388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdbd87-4c57-4a1d-b548-db875aaf13e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:14.394302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b6fdbd87-4c57-4a1d-b548-db875aaf13e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:14.502438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdbd87-4c57-4a1d-b548-db875aaf13e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:14.503855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdbd87-4c57-4a1d-b548-db875aaf13e7 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:14.527457 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 669/1316] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:37:17.080038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebd679-8a93-4408-a0e4-d91383e46506 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:17.080566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebd679-8a93-4408-a0e4-d91383e46506 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:17.080566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebd679-8a93-4408-a0e4-d91383e46506 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:17.082685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 648/1317] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:17.087612 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bfb5f-0f2d-42b4-8c12-7243240b0a48 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:17.088044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bfb5f-0f2d-42b4-8c12-7243240b0a48 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:17.088183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bfb5f-0f2d-42b4-8c12-7243240b0a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:17.156117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-300bfb5f-0f2d-42b4-8c12-7243240b0a48 None None] MFA Rules not processed for user `9dcd945ca11c4739972e841baad81073`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:17.157382 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-300bfb5f-0f2d-42b4-8c12-7243240b0a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:17.209301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:17.253632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 670/1318] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:17 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:17.294492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebd679-8a93-4408-a0e4-d91383e46506 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:17.296024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b0ebd679-8a93-4408-a0e4-d91383e46506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:17.398413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebd679-8a93-4408-a0e4-d91383e46506 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:17.401249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:17.469650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c07d8-de29-4634-a026-ce4300f1c000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:17.470297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c07d8-de29-4634-a026-ce4300f1c000 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:17.470580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c07d8-de29-4634-a026-ce4300f1c000 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:17.470855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c07d8-de29-4634-a026-ce4300f1c000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:17.472498 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-075c07d8-de29-4634-a026-ce4300f1c000 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:17.577448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c07d8-de29-4634-a026-ce4300f1c000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:17.578871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c07d8-de29-4634-a026-ce4300f1c000 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:17.597558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 649/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:17 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:37:20.318368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be351851-2cf9-458f-a5f1-19fda3b4b98c None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:20.318830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be351851-2cf9-458f-a5f1-19fda3b4b98c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:20.318948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be351851-2cf9-458f-a5f1-19fda3b4b98c None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:20.320315 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 671/1320] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:20.325551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115d1dc-518a-4252-99ee-88f2aa916fb8 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:20.325961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115d1dc-518a-4252-99ee-88f2aa916fb8 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:20.326120 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115d1dc-518a-4252-99ee-88f2aa916fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:20.366819 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:20.393166 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-c115d1dc-518a-4252-99ee-88f2aa916fb8 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:20.394928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c115d1dc-518a-4252-99ee-88f2aa916fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:20.489962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 650/1321] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:20.528873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be351851-2cf9-458f-a5f1-19fda3b4b98c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:20.530393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-be351851-2cf9-458f-a5f1-19fda3b4b98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:20.632392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be351851-2cf9-458f-a5f1-19fda3b4b98c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:20.705103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c7ec3-56c7-4e61-bcb3-a67225a9f48c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:20.705749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c7ec3-56c7-4e61-bcb3-a67225a9f48c service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:20.706038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c7ec3-56c7-4e61-bcb3-a67225a9f48c service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:20.706319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c7ec3-56c7-4e61-bcb3-a67225a9f48c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:20.708553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2e3c7ec3-56c7-4e61-bcb3-a67225a9f48c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:20.715903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:20.818096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c7ec3-56c7-4e61-bcb3-a67225a9f48c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:20.819526 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c7ec3-56c7-4e61-bcb3-a67225a9f48c service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:20.837643 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 672/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:37:23.464259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980e764-313d-4cf0-9bcb-c34e2967c1f9 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:23.464837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980e764-313d-4cf0-9bcb-c34e2967c1f9 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:23.464837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980e764-313d-4cf0-9bcb-c34e2967c1f9 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:23.466260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 651/1323] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:23.471565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118d3a0-3780-4478-92a4-22ae73918e7f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:23.472001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118d3a0-3780-4478-92a4-22ae73918e7f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:23.472146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118d3a0-3780-4478-92a4-22ae73918e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:23.537179 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4118d3a0-3780-4478-92a4-22ae73918e7f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:23.538379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4118d3a0-3780-4478-92a4-22ae73918e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:23.595599 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:23.640672 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 673/1324] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:23.672708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980e764-313d-4cf0-9bcb-c34e2967c1f9 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:23.674278 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0980e764-313d-4cf0-9bcb-c34e2967c1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:23.784038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980e764-313d-4cf0-9bcb-c34e2967c1f9 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:23.797001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:23.861244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5a8c5-ad2d-4add-8cb0-2493af988bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:23.861820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5a8c5-ad2d-4add-8cb0-2493af988bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:23.862111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5a8c5-ad2d-4add-8cb0-2493af988bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:23.862391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5a8c5-ad2d-4add-8cb0-2493af988bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:23.864992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f6e5a8c5-ad2d-4add-8cb0-2493af988bf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:23.973356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5a8c5-ad2d-4add-8cb0-2493af988bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:23.974789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5a8c5-ad2d-4add-8cb0-2493af988bf2 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:23.992945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 652/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:37:26.011959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6b648-32cd-4045-ac85-4df6143988cc None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:26.012490 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6b648-32cd-4045-ac85-4df6143988cc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:26.012490 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6b648-32cd-4045-ac85-4df6143988cc None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:26.013926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 674/1326] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:26.019867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a5392-7daf-4384-bd16-9285570b86ae None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:26.020271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a5392-7daf-4384-bd16-9285570b86ae None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:26.020497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a5392-7daf-4384-bd16-9285570b86ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:26.072549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:26.091405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-d43a5392-7daf-4384-bd16-9285570b86ae None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:26.092594 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d43a5392-7daf-4384-bd16-9285570b86ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:26.187961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 653/1327] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:26.223847 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6b648-32cd-4045-ac85-4df6143988cc None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:26.225449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2cf6b648-32cd-4045-ac85-4df6143988cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:26.331943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6b648-32cd-4045-ac85-4df6143988cc None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:26.408924 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c242e-05d4-4f96-81d5-422c6d45bdf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:26.409590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c242e-05d4-4f96-81d5-422c6d45bdf5 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:26.409875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c242e-05d4-4f96-81d5-422c6d45bdf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:26.410155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c242e-05d4-4f96-81d5-422c6d45bdf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:26.411806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d18c242e-05d4-4f96-81d5-422c6d45bdf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:26.429156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:26.527269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c242e-05d4-4f96-81d5-422c6d45bdf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:26.528749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c242e-05d4-4f96-81d5-422c6d45bdf5 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:26.548227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 675/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:37:28.590476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e6085-9629-4dee-a063-c38044f43e75 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:28.591000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e6085-9629-4dee-a063-c38044f43e75 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:28.591000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e6085-9629-4dee-a063-c38044f43e75 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:28.592387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 654/1329] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:28.598783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bf729-5633-4a91-a93a-cdf47c57c539 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:28.599312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bf729-5633-4a91-a93a-cdf47c57c539 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:28.599312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bf729-5633-4a91-a93a-cdf47c57c539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:28.665872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-855bf729-5633-4a91-a93a-cdf47c57c539 None None] MFA Rules not processed for user `9dcd945ca11c4739972e841baad81073`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:28.667090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-855bf729-5633-4a91-a93a-cdf47c57c539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:28.730648 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:28.762653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 676/1330] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:28 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:28.800383 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e6085-9629-4dee-a063-c38044f43e75 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:28.801991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-859e6085-9629-4dee-a063-c38044f43e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:28.910821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e6085-9629-4dee-a063-c38044f43e75 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:28.930325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:28.984315 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e283f4-d9db-4dba-b90d-de5ea403b7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:28.985005 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e283f4-d9db-4dba-b90d-de5ea403b7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:28.985290 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e283f4-d9db-4dba-b90d-de5ea403b7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:28.985566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e283f4-d9db-4dba-b90d-de5ea403b7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:28.987218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-07e283f4-d9db-4dba-b90d-de5ea403b7c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:29.096412 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e283f4-d9db-4dba-b90d-de5ea403b7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:29.097879 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e283f4-d9db-4dba-b90d-de5ea403b7c9 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:29.116887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 655/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:28 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:37:31.360677 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9e86f-46a5-409c-9c55-072ec8457d7d None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:31.361155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9e86f-46a5-409c-9c55-072ec8457d7d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:31.361271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9e86f-46a5-409c-9c55-072ec8457d7d None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:31.363107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 677/1332] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:31.368133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009da62a-3c12-437b-99b0-93f15bc3fa35 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:31.368547 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009da62a-3c12-437b-99b0-93f15bc3fa35 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:31.368724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009da62a-3c12-437b-99b0-93f15bc3fa35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:31.431694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:31.434728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-009da62a-3c12-437b-99b0-93f15bc3fa35 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:31.436646 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-009da62a-3c12-437b-99b0-93f15bc3fa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:31.530971 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 656/1333] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:31.567222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9e86f-46a5-409c-9c55-072ec8457d7d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:31.568767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d7e9e86f-46a5-409c-9c55-072ec8457d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:31.669243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9e86f-46a5-409c-9c55-072ec8457d7d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:31.741004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a35f8a-81dc-468a-b086-25be75f36952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:31.741652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a35f8a-81dc-468a-b086-25be75f36952 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:31.741937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a35f8a-81dc-468a-b086-25be75f36952 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:31.742217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a35f8a-81dc-468a-b086-25be75f36952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:31.743842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c2a35f8a-81dc-468a-b086-25be75f36952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:31.770541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:31.850301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a35f8a-81dc-468a-b086-25be75f36952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:31.851724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a35f8a-81dc-468a-b086-25be75f36952 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:31.870242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 678/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:37:33.903183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff8fd5-d55c-4898-ad3e-62c6bad9d36f None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:33.903828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff8fd5-d55c-4898-ad3e-62c6bad9d36f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:33.903828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff8fd5-d55c-4898-ad3e-62c6bad9d36f None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:33.905165 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 657/1335] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:33.910272 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f828c-799c-4250-80e1-6129b873559e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:33.910653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f828c-799c-4250-80e1-6129b873559e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:33.910841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f828c-799c-4250-80e1-6129b873559e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:33.976951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-9f9f828c-799c-4250-80e1-6129b873559e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:33.978188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9f9f828c-799c-4250-80e1-6129b873559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:34.051637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:34.074545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 679/1336] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:34.114659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff8fd5-d55c-4898-ad3e-62c6bad9d36f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:34.116170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a1ff8fd5-d55c-4898-ad3e-62c6bad9d36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:34.218646 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff8fd5-d55c-4898-ad3e-62c6bad9d36f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:34.245991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:34.290662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb91cc-42a3-49c8-97ec-28b8ca4fe329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:34.291310 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb91cc-42a3-49c8-97ec-28b8ca4fe329 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:34.291591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb91cc-42a3-49c8-97ec-28b8ca4fe329 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:34.291865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb91cc-42a3-49c8-97ec-28b8ca4fe329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:34.293546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-13fb91cc-42a3-49c8-97ec-28b8ca4fe329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:34.408270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb91cc-42a3-49c8-97ec-28b8ca4fe329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:34.409789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb91cc-42a3-49c8-97ec-28b8ca4fe329 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:34.428259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 658/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:37:36.763990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49618a4-0b26-4db5-9649-2cfb9132b19d None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:36.764632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49618a4-0b26-4db5-9649-2cfb9132b19d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:36.764632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49618a4-0b26-4db5-9649-2cfb9132b19d None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:36.765999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 680/1338] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:36.771422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aefff-7f80-4808-8b48-5ca42fada937 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:36.771823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aefff-7f80-4808-8b48-5ca42fada937 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:36.771972 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aefff-7f80-4808-8b48-5ca42fada937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:36.835633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-f34aefff-7f80-4808-8b48-5ca42fada937 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:36.836851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f34aefff-7f80-4808-8b48-5ca42fada937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:36.844888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:36.933381 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 659/1339] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:36.968940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49618a4-0b26-4db5-9649-2cfb9132b19d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:36.970461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c49618a4-0b26-4db5-9649-2cfb9132b19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:37.074600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49618a4-0b26-4db5-9649-2cfb9132b19d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:37.146470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78509a9-130c-4d8b-9a73-c313b7fc17ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:37.147112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78509a9-130c-4d8b-9a73-c313b7fc17ef service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:37.147394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78509a9-130c-4d8b-9a73-c313b7fc17ef service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:37.147663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78509a9-130c-4d8b-9a73-c313b7fc17ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:37.149319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b78509a9-130c-4d8b-9a73-c313b7fc17ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:37.184237 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:37.255729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78509a9-130c-4d8b-9a73-c313b7fc17ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:37.257207 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78509a9-130c-4d8b-9a73-c313b7fc17ef service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:37.276239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 681/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:37:41.131897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6832d-64a2-4467-9e71-041bdf3a028b None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:41.132492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6832d-64a2-4467-9e71-041bdf3a028b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:41.132492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6832d-64a2-4467-9e71-041bdf3a028b None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:41.133868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 660/1341] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:41.139068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb557143-e41f-4531-9fb6-c4e11a28b7a2 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:41.139458 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb557143-e41f-4531-9fb6-c4e11a28b7a2 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:41.139620 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb557143-e41f-4531-9fb6-c4e11a28b7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:41.208639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-fb557143-e41f-4531-9fb6-c4e11a28b7a2 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:41.209848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fb557143-e41f-4531-9fb6-c4e11a28b7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:41.297788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:41.311598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 682/1342] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:41.351149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6832d-64a2-4467-9e71-041bdf3a028b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:41.352616 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5da6832d-64a2-4467-9e71-041bdf3a028b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:41.453387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6832d-64a2-4467-9e71-041bdf3a028b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:41.491618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:41.528807 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf6fba-5133-4ca2-8570-0f760a808df4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:41.529459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf6fba-5133-4ca2-8570-0f760a808df4 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:41.529745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf6fba-5133-4ca2-8570-0f760a808df4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:41.530022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf6fba-5133-4ca2-8570-0f760a808df4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:41.531674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c1bf6fba-5133-4ca2-8570-0f760a808df4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:41.638306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf6fba-5133-4ca2-8570-0f760a808df4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:41.639754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf6fba-5133-4ca2-8570-0f760a808df4 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:41.660279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 661/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:37:44.291991 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01af5ab-cabe-4638-b0cb-492037aecf07 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:44.292531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01af5ab-cabe-4638-b0cb-492037aecf07 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:44.292531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01af5ab-cabe-4638-b0cb-492037aecf07 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:44.294020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 683/1344] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:44.300392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004382f1-8502-4ed8-9280-52dc164c36b1 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:44.300945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004382f1-8502-4ed8-9280-52dc164c36b1 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:44.301063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004382f1-8502-4ed8-9280-52dc164c36b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:44.372151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-004382f1-8502-4ed8-9280-52dc164c36b1 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:44.373405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-004382f1-8502-4ed8-9280-52dc164c36b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:44.393599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:44.476162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 662/1345] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:44.513281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01af5ab-cabe-4638-b0cb-492037aecf07 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:44.514921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d01af5ab-cabe-4638-b0cb-492037aecf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:44.619036 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01af5ab-cabe-4638-b0cb-492037aecf07 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:44.692115 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48924108-b689-4326-9c8f-49d5c4ac9fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:44.692725 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48924108-b689-4326-9c8f-49d5c4ac9fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:44.693008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48924108-b689-4326-9c8f-49d5c4ac9fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:44.693279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48924108-b689-4326-9c8f-49d5c4ac9fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:44.694910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-48924108-b689-4326-9c8f-49d5c4ac9fa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:44.738087 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:44.805853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48924108-b689-4326-9c8f-49d5c4ac9fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:44.807279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48924108-b689-4326-9c8f-49d5c4ac9fa9 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:44.826139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 684/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:37:48.734537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06d001-be7d-45fc-9a6c-4b4eebe4c89c None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:48.735066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06d001-be7d-45fc-9a6c-4b4eebe4c89c None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:48.735066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06d001-be7d-45fc-9a6c-4b4eebe4c89c None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:48.736474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 663/1347] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:48.741652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fbfd6-72e9-4504-a056-007b35cfdd63 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:48.742067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fbfd6-72e9-4504-a056-007b35cfdd63 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:48.742206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fbfd6-72e9-4504-a056-007b35cfdd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:48.809896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-dd2fbfd6-72e9-4504-a056-007b35cfdd63 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:48.811335 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-dd2fbfd6-72e9-4504-a056-007b35cfdd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:48.909725 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:48.916325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 685/1348] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:48.950968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06d001-be7d-45fc-9a6c-4b4eebe4c89c None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:37:48.952698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ba06d001-be7d-45fc-9a6c-4b4eebe4c89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:49.082046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06d001-be7d-45fc-9a6c-4b4eebe4c89c None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:37:49.132014 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:49.160297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4cd91-7508-4ce5-a7b3-e457e23c2a1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:37:49.161009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4cd91-7508-4ce5-a7b3-e457e23c2a1a service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:49.161300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4cd91-7508-4ce5-a7b3-e457e23c2a1a service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:49.161579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4cd91-7508-4ce5-a7b3-e457e23c2a1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:49.163290 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b3f4cd91-7508-4ce5-a7b3-e457e23c2a1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:49.278111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4cd91-7508-4ce5-a7b3-e457e23c2a1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:37:49.279599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4cd91-7508-4ce5-a7b3-e457e23c2a1a service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:37:49.300071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 664/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:37:54.139451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b1c0a-065b-4d5b-8118-cca29a46eea0 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:54.139961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b1c0a-065b-4d5b-8118-cca29a46eea0 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:54.140059 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b1c0a-065b-4d5b-8118-cca29a46eea0 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:54.142573 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 686/1350] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:54.148201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c0819-23bf-41c6-b12a-33029184bca7 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:54.148665 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c0819-23bf-41c6-b12a-33029184bca7 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:54.149158 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c0819-23bf-41c6-b12a-33029184bca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:54.284653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-807c0819-23bf-41c6-b12a-33029184bca7 None None] MFA Rules not processed for user `62bf4ef4fce74f7b91e453f73430d9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:54.285762 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-807c0819-23bf-41c6-b12a-33029184bca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:54.305819 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:54.404625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 665/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:37:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:58.585097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fc802-8106-44a9-b361-df6e498c98a8 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:58.585601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fc802-8106-44a9-b361-df6e498c98a8 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:58.585601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fc802-8106-44a9-b361-df6e498c98a8 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:58.587228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 687/1352] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:58.592363 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a191b3-95ab-44a3-a2ac-fae87be2ca80 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:58.592811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a191b3-95ab-44a3-a2ac-fae87be2ca80 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:58.592991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a191b3-95ab-44a3-a2ac-fae87be2ca80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:58.682418 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-60a191b3-95ab-44a3-a2ac-fae87be2ca80 None None] MFA Rules not processed for user `62bf4ef4fce74f7b91e453f73430d9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:37:58.683974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-60a191b3-95ab-44a3-a2ac-fae87be2ca80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:37:58.762559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:37:58.803243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 666/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:37:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:37:59.971727 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b96c5-e0f9-4569-8441-13d83d2eeefe None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:59.972209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b96c5-e0f9-4569-8441-13d83d2eeefe None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:59.972209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b96c5-e0f9-4569-8441-13d83d2eeefe None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:37:59.973592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 688/1354] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:37:59.978505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b972d-365d-4e41-b2e2-1893bb493013 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:37:59.978952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b972d-365d-4e41-b2e2-1893bb493013 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:37:59.979377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b972d-365d-4e41-b2e2-1893bb493013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:00.056540 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-821b972d-365d-4e41-b2e2-1893bb493013 None None] MFA Rules not processed for user `62bf4ef4fce74f7b91e453f73430d9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:00.057767 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-821b972d-365d-4e41-b2e2-1893bb493013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:00.164141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 667/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:37:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:38:03.560749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4e250-f9b0-4b8d-8054-e0c2a040634b None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:03.561834 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4e250-f9b0-4b8d-8054-e0c2a040634b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:03.562163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4e250-f9b0-4b8d-8054-e0c2a040634b None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:03.563592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 689/1356] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:03.569876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe98ec-1664-49ba-b135-050237911841 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:03.570373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe98ec-1664-49ba-b135-050237911841 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:03.570473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe98ec-1664-49ba-b135-050237911841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:03.619958 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:03.666476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-b4fe98ec-1664-49ba-b135-050237911841 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:03.668125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b4fe98ec-1664-49ba-b135-050237911841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:03.775363 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 668/1357] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:03.830913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59929d-546b-4ff3-92f2-b661c64e2778 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:03.831380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59929d-546b-4ff3-92f2-b661c64e2778 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:03.831933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59929d-546b-4ff3-92f2-b661c64e2778 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:03.834116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 690/1358] 10.222.0.52 () {54 vars in 962 bytes} [Mon Jul 25 08:38:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:03.842356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bed83f-ce0f-4013-a376-8a64e970dc98 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:03.842886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bed83f-ce0f-4013-a376-8a64e970dc98 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:03.843458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bed83f-ce0f-4013-a376-8a64e970dc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:03.930958 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-29bed83f-ce0f-4013-a376-8a64e970dc98 None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:03.932141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-29bed83f-ce0f-4013-a376-8a64e970dc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:03.982464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:04.056282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 669/1359] 10.222.0.52 () {58 vars in 1059 bytes} [Mon Jul 25 08:38:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:38:04.064011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59929d-546b-4ff3-92f2-b661c64e2778 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:04.065779 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-bb59929d-546b-4ff3-92f2-b661c64e2778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:04.219231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59929d-546b-4ff3-92f2-b661c64e2778 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:04.297445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:04.341562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432c0eb-fe37-4996-85d3-fe9861b5b46e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:04.342223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432c0eb-fe37-4996-85d3-fe9861b5b46e service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:04.342514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432c0eb-fe37-4996-85d3-fe9861b5b46e service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:04.342792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432c0eb-fe37-4996-85d3-fe9861b5b46e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:04.344519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b432c0eb-fe37-4996-85d3-fe9861b5b46e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:04.445911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432c0eb-fe37-4996-85d3-fe9861b5b46e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:04.447530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432c0eb-fe37-4996-85d3-fe9861b5b46e service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:04.467429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 691/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:06.659255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cb248-b759-4a8a-8c97-0a0c7bf75daa None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:06.659816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cb248-b759-4a8a-8c97-0a0c7bf75daa None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:06.659816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cb248-b759-4a8a-8c97-0a0c7bf75daa None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:06.664620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 670/1361] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:06.666729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbc5ed-6855-483f-a4a6-9d4b2bbc1384 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:06.667141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbc5ed-6855-483f-a4a6-9d4b2bbc1384 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:06.667316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbc5ed-6855-483f-a4a6-9d4b2bbc1384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:06.742423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-b7bbc5ed-6855-483f-a4a6-9d4b2bbc1384 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:06.743912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b7bbc5ed-6855-483f-a4a6-9d4b2bbc1384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:06.787854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:06.854601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 692/1362] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:06.899977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cb248-b759-4a8a-8c97-0a0c7bf75daa None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:06.901848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-077cb248-b759-4a8a-8c97-0a0c7bf75daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:07.080492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cb248-b759-4a8a-8c97-0a0c7bf75daa None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:07.090774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:07.235076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937b5fa-5096-4e51-89aa-7e63af2f7791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:07.236903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937b5fa-5096-4e51-89aa-7e63af2f7791 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:07.236903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937b5fa-5096-4e51-89aa-7e63af2f7791 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:07.237152 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937b5fa-5096-4e51-89aa-7e63af2f7791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:07.238870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-9937b5fa-5096-4e51-89aa-7e63af2f7791 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:07.340208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937b5fa-5096-4e51-89aa-7e63af2f7791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:07.341927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937b5fa-5096-4e51-89aa-7e63af2f7791 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:07.361267 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 671/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:09.725736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd8ed6-2594-44a8-9242-c5633cb655a9 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:09.726270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd8ed6-2594-44a8-9242-c5633cb655a9 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:09.726270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd8ed6-2594-44a8-9242-c5633cb655a9 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:09.728922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 693/1364] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:09.733533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26fdd1-3bf7-4f1f-bd28-caaf5433df56 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:09.733982 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26fdd1-3bf7-4f1f-bd28-caaf5433df56 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:09.734336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26fdd1-3bf7-4f1f-bd28-caaf5433df56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:09.756687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:09.823688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-6d26fdd1-3bf7-4f1f-bd28-caaf5433df56 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:09.824923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-6d26fdd1-3bf7-4f1f-bd28-caaf5433df56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:09.930053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 672/1365] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:09.978158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f9621-7352-4cbd-bd6e-f59e21554250 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:09.978453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f9621-7352-4cbd-bd6e-f59e21554250 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:09.978734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f9621-7352-4cbd-bd6e-f59e21554250 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:09.980112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 694/1366] 10.222.0.52 () {54 vars in 962 bytes} [Mon Jul 25 08:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:09.986251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a43104-f992-432e-a569-4425d8e58d4e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:09.986906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a43104-f992-432e-a569-4425d8e58d4e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:09.987294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a43104-f992-432e-a569-4425d8e58d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:10.191492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-a7a43104-f992-432e-a569-4425d8e58d4e None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:10.192645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a7a43104-f992-432e-a569-4425d8e58d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:10.229966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:10.326419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 673/1367] 10.222.0.52 () {58 vars in 1059 bytes} [Mon Jul 25 08:38:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:10.335240 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f9621-7352-4cbd-bd6e-f59e21554250 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:10.336787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-719f9621-7352-4cbd-bd6e-f59e21554250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:10.612152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f9621-7352-4cbd-bd6e-f59e21554250 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:10.633502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:10.736121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4851f85-e3d6-4e32-92be-ad76c2d5e829 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:10.736995 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851f85-e3d6-4e32-92be-ad76c2d5e829 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:10.737140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851f85-e3d6-4e32-92be-ad76c2d5e829 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:10.737385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851f85-e3d6-4e32-92be-ad76c2d5e829 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:10.739156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a4851f85-e3d6-4e32-92be-ad76c2d5e829 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:10.834103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4851f85-e3d6-4e32-92be-ad76c2d5e829 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:10.835698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4851f85-e3d6-4e32-92be-ad76c2d5e829 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:10.854578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 695/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:11.638305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d133f5-5697-4f24-8ad8-395500322c19 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:11.638820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d133f5-5697-4f24-8ad8-395500322c19 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:11.638903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d133f5-5697-4f24-8ad8-395500322c19 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:11.640611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 674/1369] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:38:11.646321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17008e-c2c3-4e23-893d-f63d73f45718 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:11.646610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17008e-c2c3-4e23-893d-f63d73f45718 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:11.646886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17008e-c2c3-4e23-893d-f63d73f45718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:11.680741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:11.717604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-6b17008e-c2c3-4e23-893d-f63d73f45718 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:11.718823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6b17008e-c2c3-4e23-893d-f63d73f45718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:11.822074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 696/1370] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:12.167926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f58de-21f5-4872-a7c3-892460874284 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:12.167926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f58de-21f5-4872-a7c3-892460874284 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:12.167926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f58de-21f5-4872-a7c3-892460874284 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:12.167926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 675/1371] 10.222.0.52 () {54 vars in 964 bytes} [Mon Jul 25 08:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:12.174805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef77fe2-951f-43ef-a029-7c0b702416cc None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:12.175163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef77fe2-951f-43ef-a029-7c0b702416cc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:12.175514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef77fe2-951f-43ef-a029-7c0b702416cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:12.256198 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-9ef77fe2-951f-43ef-a029-7c0b702416cc None None] MFA Rules not processed for user `84871b711bcb47efb039ce40766cf3db`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:12.261352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9ef77fe2-951f-43ef-a029-7c0b702416cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:12.330797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:12.361647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 697/1372] 10.222.0.52 () {58 vars in 1061 bytes} [Mon Jul 25 08:38:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:38:12.368884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f58de-21f5-4872-a7c3-892460874284 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:12.370372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-343f58de-21f5-4872-a7c3-892460874284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:12.483292 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f58de-21f5-4872-a7c3-892460874284 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:12.529308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:12.569950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46aac2-6098-48e6-8799-1a14ab13a89f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:12.570641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46aac2-6098-48e6-8799-1a14ab13a89f service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:12.570921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46aac2-6098-48e6-8799-1a14ab13a89f service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:12.571198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46aac2-6098-48e6-8799-1a14ab13a89f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:12.572857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-cc46aac2-6098-48e6-8799-1a14ab13a89f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:12.694727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46aac2-6098-48e6-8799-1a14ab13a89f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:12.696265 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46aac2-6098-48e6-8799-1a14ab13a89f service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:12.732780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 676/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:12.916140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b2b17-1cea-41c2-b85b-c9081a1aa7cd None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:12.916616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b2b17-1cea-41c2-b85b-c9081a1aa7cd None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:12.916756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b2b17-1cea-41c2-b85b-c9081a1aa7cd None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:12.918157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 698/1374] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:12.923226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77346a21-d9a5-4617-a33d-f20f635e50b2 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:12.923564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77346a21-d9a5-4617-a33d-f20f635e50b2 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:12.923783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77346a21-d9a5-4617-a33d-f20f635e50b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:12.994078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-77346a21-d9a5-4617-a33d-f20f635e50b2 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:12.995243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-77346a21-d9a5-4617-a33d-f20f635e50b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:13.026388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:13.102364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 677/1375] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:13.147350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b2b17-1cea-41c2-b85b-c9081a1aa7cd None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:13.155309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d78b2b17-1cea-41c2-b85b-c9081a1aa7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:13.306247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b2b17-1cea-41c2-b85b-c9081a1aa7cd None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:13.358017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:13.425499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9a847-117b-4eec-ae15-94d111ec0a7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:13.426177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9a847-117b-4eec-ae15-94d111ec0a7e service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:13.426466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9a847-117b-4eec-ae15-94d111ec0a7e service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:13.426740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9a847-117b-4eec-ae15-94d111ec0a7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:13.428497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-68d9a847-117b-4eec-ae15-94d111ec0a7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:13.517711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9a847-117b-4eec-ae15-94d111ec0a7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:13.519323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9a847-117b-4eec-ae15-94d111ec0a7e service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:13.536915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 699/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:15.555608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fccca-e1f8-4ad0-a71b-4b59d47488ca None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:15.556176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fccca-e1f8-4ad0-a71b-4b59d47488ca None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:15.556176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fccca-e1f8-4ad0-a71b-4b59d47488ca None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:15.557599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 678/1377] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:15.562759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd4886-d13a-45ee-909d-547415522dcc None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:15.563181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd4886-d13a-45ee-909d-547415522dcc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:15.563454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd4886-d13a-45ee-909d-547415522dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:15.636293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-1fbd4886-d13a-45ee-909d-547415522dcc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:15.637590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1fbd4886-d13a-45ee-909d-547415522dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:15.645436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:15.754067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 700/1378] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:15.797795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fccca-e1f8-4ad0-a71b-4b59d47488ca None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:15.799310 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ba0fccca-e1f8-4ad0-a71b-4b59d47488ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:15.942824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fccca-e1f8-4ad0-a71b-4b59d47488ca None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:15.946590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:16.068046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69273fdd-3ff0-4cdb-b36a-62283c6d68a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:16.068823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69273fdd-3ff0-4cdb-b36a-62283c6d68a7 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:16.069171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69273fdd-3ff0-4cdb-b36a-62283c6d68a7 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:16.069519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69273fdd-3ff0-4cdb-b36a-62283c6d68a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:16.071504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-69273fdd-3ff0-4cdb-b36a-62283c6d68a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:16.179090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69273fdd-3ff0-4cdb-b36a-62283c6d68a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:16.180974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69273fdd-3ff0-4cdb-b36a-62283c6d68a7 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:16.207838 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 679/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:18.262650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbb97b-10ec-49fd-a8d7-8cdb04487f2d None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:18.263226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbb97b-10ec-49fd-a8d7-8cdb04487f2d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:18.263226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbb97b-10ec-49fd-a8d7-8cdb04487f2d None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:18.265014 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 701/1380] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:18.270051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6344f-c966-472d-b5db-fbfce5432e2c None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:18.270456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6344f-c966-472d-b5db-fbfce5432e2c None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:18.270664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6344f-c966-472d-b5db-fbfce5432e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:18.282566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:18.353964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-63d6344f-c966-472d-b5db-fbfce5432e2c None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:18.355169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-63d6344f-c966-472d-b5db-fbfce5432e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:18.457750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 680/1381] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:18.501591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbb97b-10ec-49fd-a8d7-8cdb04487f2d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:18.503262 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d0bbb97b-10ec-49fd-a8d7-8cdb04487f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:18.656176 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:18.668470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbb97b-10ec-49fd-a8d7-8cdb04487f2d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:18.789447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bde7b-1be2-4935-a557-e76853834adc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:18.790185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bde7b-1be2-4935-a557-e76853834adc service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:18.790582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bde7b-1be2-4935-a557-e76853834adc service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:18.790990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bde7b-1be2-4935-a557-e76853834adc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:18.792778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-280bde7b-1be2-4935-a557-e76853834adc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:18.891318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bde7b-1be2-4935-a557-e76853834adc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:18.893128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bde7b-1be2-4935-a557-e76853834adc service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:18.904155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:18.914820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 702/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:21.203099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b6630-957c-4578-87d2-194d30fbf8bf None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:21.204244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b6630-957c-4578-87d2-194d30fbf8bf None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:21.204374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b6630-957c-4578-87d2-194d30fbf8bf None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:21.206364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 681/1383] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:21.212631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e4585-20e0-4426-87ec-9921c87a1779 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:21.213047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e4585-20e0-4426-87ec-9921c87a1779 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:21.213271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e4585-20e0-4426-87ec-9921c87a1779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:21.282289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-977e4585-20e0-4426-87ec-9921c87a1779 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:21.283579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-977e4585-20e0-4426-87ec-9921c87a1779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:21.400463 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 703/1384] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:21.454325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b6630-957c-4578-87d2-194d30fbf8bf None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:21.456318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d34b6630-957c-4578-87d2-194d30fbf8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:21.544906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:21.606615 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b6630-957c-4578-87d2-194d30fbf8bf None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:21.730515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d587527-3acc-4e8e-8564-2d1c84fa8064 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:21.731270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d587527-3acc-4e8e-8564-2d1c84fa8064 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:21.731642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d587527-3acc-4e8e-8564-2d1c84fa8064 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:21.732061 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d587527-3acc-4e8e-8564-2d1c84fa8064 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:21.734261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4d587527-3acc-4e8e-8564-2d1c84fa8064 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:21.798603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:21.838724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d587527-3acc-4e8e-8564-2d1c84fa8064 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:21.840322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d587527-3acc-4e8e-8564-2d1c84fa8064 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:21.863445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 682/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:24.003043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f992398-1c23-4c63-b4fd-e20468012fbc None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:24.003799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f992398-1c23-4c63-b4fd-e20468012fbc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:24.003799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f992398-1c23-4c63-b4fd-e20468012fbc None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:24.005037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 704/1386] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:24.010886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b7936-04b8-41e1-8b92-a7d21f9f3fe5 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:24.011332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b7936-04b8-41e1-8b92-a7d21f9f3fe5 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:24.011615 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b7936-04b8-41e1-8b92-a7d21f9f3fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:24.098831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-9b0b7936-04b8-41e1-8b92-a7d21f9f3fe5 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:24.100921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-9b0b7936-04b8-41e1-8b92-a7d21f9f3fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:24.217517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 683/1387] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:24.263290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f992398-1c23-4c63-b4fd-e20468012fbc None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:24.265622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0f992398-1c23-4c63-b4fd-e20468012fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:24.370333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:24.442497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f992398-1c23-4c63-b4fd-e20468012fbc None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:24.557411 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee296b-9bf8-4d3a-98e6-a751a81f68c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:24.557997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee296b-9bf8-4d3a-98e6-a751a81f68c2 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:24.558287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee296b-9bf8-4d3a-98e6-a751a81f68c2 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:24.558572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee296b-9bf8-4d3a-98e6-a751a81f68c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:24.560272 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-81ee296b-9bf8-4d3a-98e6-a751a81f68c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:24.600336 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:24.654886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee296b-9bf8-4d3a-98e6-a751a81f68c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:24.656504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee296b-9bf8-4d3a-98e6-a751a81f68c2 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:24.675593 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 705/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:26.740638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686e17c-c4ce-46ee-8bc4-d69de1ba7746 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:26.741169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686e17c-c4ce-46ee-8bc4-d69de1ba7746 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:26.741282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686e17c-c4ce-46ee-8bc4-d69de1ba7746 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:26.742760 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 684/1389] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:26.747725 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aae422-23f2-4310-b4ed-b9fb7771b0bb None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:26.748148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aae422-23f2-4310-b4ed-b9fb7771b0bb None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:26.748373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aae422-23f2-4310-b4ed-b9fb7771b0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:26.815255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-21aae422-23f2-4310-b4ed-b9fb7771b0bb None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:26.816464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-21aae422-23f2-4310-b4ed-b9fb7771b0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:26.927038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 706/1390] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:26.975057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686e17c-c4ce-46ee-8bc4-d69de1ba7746 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:26.976653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4686e17c-c4ce-46ee-8bc4-d69de1ba7746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:27.007956 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:27.122705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686e17c-c4ce-46ee-8bc4-d69de1ba7746 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:27.223254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:27.236623 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e40bf-4424-4cfa-ad69-1f0dbd184e4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:27.237273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e40bf-4424-4cfa-ad69-1f0dbd184e4e service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:27.237552 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e40bf-4424-4cfa-ad69-1f0dbd184e4e service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:27.237821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e40bf-4424-4cfa-ad69-1f0dbd184e4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:27.239473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-273e40bf-4424-4cfa-ad69-1f0dbd184e4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:27.331987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e40bf-4424-4cfa-ad69-1f0dbd184e4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:27.333602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e40bf-4424-4cfa-ad69-1f0dbd184e4e service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:27.351734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 685/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:27.753313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b61c85-d0db-4e6b-a75b-2069ef2e4cd9 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:27.753905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b61c85-d0db-4e6b-a75b-2069ef2e4cd9 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:27.753905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b61c85-d0db-4e6b-a75b-2069ef2e4cd9 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:27.755283 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 707/1392] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:27.760358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e8146-0613-47ee-ba31-c334a21983cb None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:27.760839 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e8146-0613-47ee-ba31-c334a21983cb None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:27.760955 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e8146-0613-47ee-ba31-c334a21983cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:27.828154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-441e8146-0613-47ee-ba31-c334a21983cb None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:27.830068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-441e8146-0613-47ee-ba31-c334a21983cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:27.889346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:27.928927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 686/1393] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:27.985899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff521f8-5571-4cb8-b3d5-27cc112c51e8 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:27.986186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff521f8-5571-4cb8-b3d5-27cc112c51e8 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:27.986465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff521f8-5571-4cb8-b3d5-27cc112c51e8 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:27.987855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 708/1394] 10.222.0.52 () {54 vars in 966 bytes} [Mon Jul 25 08:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:27.992956 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f01da-cf88-4158-b9c5-a693b0380672 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:27.993257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f01da-cf88-4158-b9c5-a693b0380672 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:27.993543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f01da-cf88-4158-b9c5-a693b0380672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:28.072774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-4d8f01da-cf88-4158-b9c5-a693b0380672 None None] MFA Rules not processed for user `925bf2da4df74e86aa84ec62a267d088`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:28.073916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4d8f01da-cf88-4158-b9c5-a693b0380672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:28.150539 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 687/1395] 10.222.0.52 () {58 vars in 1063 bytes} [Mon Jul 25 08:38:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:28.165023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff521f8-5571-4cb8-b3d5-27cc112c51e8 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:28.166530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-aff521f8-5571-4cb8-b3d5-27cc112c51e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:28.180500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:28.271125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff521f8-5571-4cb8-b3d5-27cc112c51e8 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:28.348937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc210b1-b35f-4d68-9da1-6779899dfd56 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '925bf2da4df74e86aa84ec62a267d088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:28.349585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc210b1-b35f-4d68-9da1-6779899dfd56 service glance] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:28.349869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc210b1-b35f-4d68-9da1-6779899dfd56 service glance] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:28.350140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc210b1-b35f-4d68-9da1-6779899dfd56 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:28.351773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-dbc210b1-b35f-4d68-9da1-6779899dfd56 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:28.421618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:28.464163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc210b1-b35f-4d68-9da1-6779899dfd56 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:28.465645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc210b1-b35f-4d68-9da1-6779899dfd56 service glance] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:28.486275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 709/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:29.222840 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317b041-34e1-4968-bf42-b5903ce135ef None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:29.223351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317b041-34e1-4968-bf42-b5903ce135ef None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:29.223565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317b041-34e1-4968-bf42-b5903ce135ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:29.289916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-0317b041-34e1-4968-bf42-b5903ce135ef None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:29.291097 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0317b041-34e1-4968-bf42-b5903ce135ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:29.299716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:29.378803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 688/1397] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:38:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:29.388068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384798a6-a755-41e1-9ee0-c2b8fd724f2e None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:29.388463 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384798a6-a755-41e1-9ee0-c2b8fd724f2e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:29.388658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384798a6-a755-41e1-9ee0-c2b8fd724f2e None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:29.390519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 710/1398] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:29.397091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028d663-35df-4b15-b57d-23ea9f57309d None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:29.397091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028d663-35df-4b15-b57d-23ea9f57309d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:29.397091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028d663-35df-4b15-b57d-23ea9f57309d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:29.421254 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72bb93-8165-4ecf-973e-6049529ff6ed None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:29.421545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72bb93-8165-4ecf-973e-6049529ff6ed None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:29.421825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72bb93-8165-4ecf-973e-6049529ff6ed None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:29.423410 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 711/1399] 10.222.0.52 () {54 vars in 943 bytes} [Mon Jul 25 08:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:38:29.429401 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62831061-0968-4e3e-96cc-fe532e2406d6 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:29.429762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62831061-0968-4e3e-96cc-fe532e2406d6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:29.430102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62831061-0968-4e3e-96cc-fe532e2406d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:29.464966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3028d663-35df-4b15-b57d-23ea9f57309d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:29.466178 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3028d663-35df-4b15-b57d-23ea9f57309d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:29.509834 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-62831061-0968-4e3e-96cc-fe532e2406d6 None None] MFA Rules not processed for user `7d6be43abd85417c9fb7bbcb36a1f77d`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:29.511032 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-62831061-0968-4e3e-96cc-fe532e2406d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:29.562655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 689/1400] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:29.607505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028d663-35df-4b15-b57d-23ea9f57309d None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:29.608740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3028d663-35df-4b15-b57d-23ea9f57309d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:29.611386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:29.620254 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 712/1401] 10.222.0.52 () {58 vars in 1040 bytes} [Mon Jul 25 08:38:29 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:38:29.630365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62831061-0968-4e3e-96cc-fe532e2406d6 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:29.634987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:29.640927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62831061-0968-4e3e-96cc-fe532e2406d6 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:29.724593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028d663-35df-4b15-b57d-23ea9f57309d None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:29.745503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac6272-96cd-4c39-b5bd-36fd547816d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:29.746161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac6272-96cd-4c39-b5bd-36fd547816d7 service swift] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:29.746446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac6272-96cd-4c39-b5bd-36fd547816d7 service swift] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:29.746727 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac6272-96cd-4c39-b5bd-36fd547816d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:29.748424 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7dac6272-96cd-4c39-b5bd-36fd547816d7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:29.832197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac6272-96cd-4c39-b5bd-36fd547816d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:29.833607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:29.833731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac6272-96cd-4c39-b5bd-36fd547816d7 service swift] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:29.836422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c6e3e-254f-491e-b10a-be319807c322 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:29.837404 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c6e3e-254f-491e-b10a-be319807c322 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:29.837930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c6e3e-254f-491e-b10a-be319807c322 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:29.838588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c6e3e-254f-491e-b10a-be319807c322 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:29.841380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ed5c6e3e-254f-491e-b10a-be319807c322 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:29.842236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 713/1402] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:38:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 08:38:29.949033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c6e3e-254f-491e-b10a-be319807c322 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:29.950949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c6e3e-254f-491e-b10a-be319807c322 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:29.977243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 690/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:38:32.020202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8799c-40f6-4c0a-a9a4-c791376f3917 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:32.020809 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8799c-40f6-4c0a-a9a4-c791376f3917 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:32.020914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8799c-40f6-4c0a-a9a4-c791376f3917 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:32.022258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 714/1404] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:32.029987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaf8a2-6ecb-4dbe-9858-9a23d534d834 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:32.029987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaf8a2-6ecb-4dbe-9858-9a23d534d834 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:32.029987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaf8a2-6ecb-4dbe-9858-9a23d534d834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:32.179740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-7ecaf8a2-6ecb-4dbe-9858-9a23d534d834 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:32.181345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7ecaf8a2-6ecb-4dbe-9858-9a23d534d834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:32.257943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:32.314238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 691/1405] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:32.348850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8799c-40f6-4c0a-a9a4-c791376f3917 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:32.350642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-77f8799c-40f6-4c0a-a9a4-c791376f3917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:32.386632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:32.511498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8799c-40f6-4c0a-a9a4-c791376f3917 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:32.626492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:32.638772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21923e36-5201-4df4-9ec0-320f24282ae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:32.639860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21923e36-5201-4df4-9ec0-320f24282ae8 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:32.640217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21923e36-5201-4df4-9ec0-320f24282ae8 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:32.640602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21923e36-5201-4df4-9ec0-320f24282ae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:32.644253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-21923e36-5201-4df4-9ec0-320f24282ae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:32.741827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21923e36-5201-4df4-9ec0-320f24282ae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:32.743470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21923e36-5201-4df4-9ec0-320f24282ae8 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:32.764621 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 715/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:38:34.823499 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76c2a0-6bcd-4c36-ba76-2267edd9350b None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:34.824048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76c2a0-6bcd-4c36-ba76-2267edd9350b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:34.824048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76c2a0-6bcd-4c36-ba76-2267edd9350b None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:34.827928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 692/1407] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:34.833532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d6269-135c-4d24-837b-fc92250cfcf1 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:34.833979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d6269-135c-4d24-837b-fc92250cfcf1 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:34.834123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d6269-135c-4d24-837b-fc92250cfcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:34.905150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-db6d6269-135c-4d24-837b-fc92250cfcf1 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:34.907006 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-db6d6269-135c-4d24-837b-fc92250cfcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:34.976908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:35.021538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 716/1408] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:38:35.067303 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76c2a0-6bcd-4c36-ba76-2267edd9350b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:35.068842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2e76c2a0-6bcd-4c36-ba76-2267edd9350b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:35.241986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76c2a0-6bcd-4c36-ba76-2267edd9350b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:35.313559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:35.378040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee2ce5-0141-4054-b785-34e6f38a5f50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:35.378710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee2ce5-0141-4054-b785-34e6f38a5f50 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:35.378999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee2ce5-0141-4054-b785-34e6f38a5f50 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:35.379280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee2ce5-0141-4054-b785-34e6f38a5f50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:35.381000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-67ee2ce5-0141-4054-b785-34e6f38a5f50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:35.504911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee2ce5-0141-4054-b785-34e6f38a5f50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:35.506488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee2ce5-0141-4054-b785-34e6f38a5f50 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:35.526802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 693/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:45.019616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=14 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:45.524334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=73 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:52.123003 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad572d-e518-4ebd-a757-4f1c34f9d21e None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:52.123548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad572d-e518-4ebd-a757-4f1c34f9d21e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:52.123548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad572d-e518-4ebd-a757-4f1c34f9d21e None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:52.124973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 717/1410] 10.222.0.52 () {54 vars in 918 bytes} [Mon Jul 25 08:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:52.130541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cf27f-59ab-4fd5-9812-e0b1ab4e1148 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:52.130969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cf27f-59ab-4fd5-9812-e0b1ab4e1148 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:52.131099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cf27f-59ab-4fd5-9812-e0b1ab4e1148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:52.206003 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-c30cf27f-59ab-4fd5-9812-e0b1ab4e1148 None None] MFA Rules not processed for user `62bf4ef4fce74f7b91e453f73430d9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:52.207193 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c30cf27f-59ab-4fd5-9812-e0b1ab4e1148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:52.310495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 694/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Mon Jul 25 08:38:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:52.325479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3cf03-5a1c-4690-b95a-22baa0406083 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:52.325774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3cf03-5a1c-4690-b95a-22baa0406083 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:52.326051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3cf03-5a1c-4690-b95a-22baa0406083 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:52.327463 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 718/1412] 10.222.0.52 () {54 vars in 918 bytes} [Mon Jul 25 08:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:38:52.331949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5851ebc-8e7e-41d9-9bbf-c89994963f31 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:52.332234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5851ebc-8e7e-41d9-9bbf-c89994963f31 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:52.332538 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5851ebc-8e7e-41d9-9bbf-c89994963f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:52.404801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:52.407701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-e5851ebc-8e7e-41d9-9bbf-c89994963f31 None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:52.409573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e5851ebc-8e7e-41d9-9bbf-c89994963f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:52.522694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 695/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Mon Jul 25 08:38:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:54.957958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe47ae-5203-497d-9632-8af758bbd9e2 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:54.958509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe47ae-5203-497d-9632-8af758bbd9e2 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:54.958509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe47ae-5203-497d-9632-8af758bbd9e2 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:54.959885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:54.965398 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625c7c9-468d-40d2-8f9e-37e5b0583d6c None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:54.965833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625c7c9-468d-40d2-8f9e-37e5b0583d6c None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:54.965963 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625c7c9-468d-40d2-8f9e-37e5b0583d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:55.030857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-0625c7c9-468d-40d2-8f9e-37e5b0583d6c None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:55.031977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0625c7c9-468d-40d2-8f9e-37e5b0583d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:55.091562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:55.132951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:55.176967 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe47ae-5203-497d-9632-8af758bbd9e2 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:55.178555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-36fe47ae-5203-497d-9632-8af758bbd9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:55.322110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe47ae-5203-497d-9632-8af758bbd9e2 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:55.393668 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:55.433502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacc67b-c972-48f3-b170-df63df6345d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:55.434152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacc67b-c972-48f3-b170-df63df6345d6 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:55.434434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacc67b-c972-48f3-b170-df63df6345d6 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:55.434708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacc67b-c972-48f3-b170-df63df6345d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:55.436339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8dacc67b-c972-48f3-b170-df63df6345d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:55.525422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacc67b-c972-48f3-b170-df63df6345d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:55.527009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacc67b-c972-48f3-b170-df63df6345d6 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:55.545496 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 720/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:56.435853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b9a0d-86bb-4b79-b26d-c662ba3e7bc9 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:56.436401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b9a0d-86bb-4b79-b26d-c662ba3e7bc9 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:56.436401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b9a0d-86bb-4b79-b26d-c662ba3e7bc9 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:56.437855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 08:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:56.443362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa804dc-e461-443a-8f64-76903139ae2d None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:56.443851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa804dc-e461-443a-8f64-76903139ae2d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:56.443962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa804dc-e461-443a-8f64-76903139ae2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:56.518835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-3aa804dc-e461-443a-8f64-76903139ae2d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:56.520033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3aa804dc-e461-443a-8f64-76903139ae2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:56.546355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:56.615777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:56.769102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b9a0d-86bb-4b79-b26d-c662ba3e7bc9 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:56.770646 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e56b9a0d-86bb-4b79-b26d-c662ba3e7bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:56.913947 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b9a0d-86bb-4b79-b26d-c662ba3e7bc9 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:56.954638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:57.026674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69320eee-1234-4a39-84c5-77726d48afd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:57.027321 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69320eee-1234-4a39-84c5-77726d48afd0 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:57.027603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69320eee-1234-4a39-84c5-77726d48afd0 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:57.027875 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69320eee-1234-4a39-84c5-77726d48afd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:57.029547 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-69320eee-1234-4a39-84c5-77726d48afd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:57.116783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69320eee-1234-4a39-84c5-77726d48afd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:57.118367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69320eee-1234-4a39-84c5-77726d48afd0 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:57.135918 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 698/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:38:58.010717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c16dbb-53cb-46c8-971a-808699f6dbdf None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:58.011406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c16dbb-53cb-46c8-971a-808699f6dbdf None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:58.011406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c16dbb-53cb-46c8-971a-808699f6dbdf None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:58.012678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 08:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:58.017794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab4781-57fd-483c-95fd-e68c0f2de111 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:58.018203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab4781-57fd-483c-95fd-e68c0f2de111 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:58.018344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab4781-57fd-483c-95fd-e68c0f2de111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:58.071559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:58.093011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-80ab4781-57fd-483c-95fd-e68c0f2de111 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:58.094201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-80ab4781-57fd-483c-95fd-e68c0f2de111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:58.189853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 08:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:58.340946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c16dbb-53cb-46c8-971a-808699f6dbdf None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:58.342474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d3c16dbb-53cb-46c8-971a-808699f6dbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:58.489334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c16dbb-53cb-46c8-971a-808699f6dbdf None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:38:58.501902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:58.602194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74302d-551f-465d-bb91-691309b665fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:38:58.602769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74302d-551f-465d-bb91-691309b665fe service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:58.603055 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74302d-551f-465d-bb91-691309b665fe service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:58.603336 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74302d-551f-465d-bb91-691309b665fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:58.605011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1e74302d-551f-465d-bb91-691309b665fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:58.693913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e74302d-551f-465d-bb91-691309b665fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:38:58.695499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e74302d-551f-465d-bb91-691309b665fe service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:38:58.713164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 723/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:38:59.572597 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f548da-9089-42ca-bc9a-ee518666a52e None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:59.573147 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f548da-9089-42ca-bc9a-ee518666a52e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:59.573147 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f548da-9089-42ca-bc9a-ee518666a52e None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:59.574493 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 08:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:38:59.579503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe435aa-6611-4eeb-b8d6-7b43dfb14396 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:38:59.579889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe435aa-6611-4eeb-b8d6-7b43dfb14396 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:38:59.580069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe435aa-6611-4eeb-b8d6-7b43dfb14396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:38:59.600194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:38:59.656542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-cfe435aa-6611-4eeb-b8d6-7b43dfb14396 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:38:59.658291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-cfe435aa-6611-4eeb-b8d6-7b43dfb14396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:38:59.753728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 08:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:38:59.904144 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f548da-9089-42ca-bc9a-ee518666a52e None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:38:59.905714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-20f548da-9089-42ca-bc9a-ee518666a52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:00.014667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:00.050073 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f548da-9089-42ca-bc9a-ee518666a52e None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:00.162450 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c59fc-8d54-4090-b386-eed460d08440 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:00.163022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c59fc-8d54-4090-b386-eed460d08440 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:00.163301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c59fc-8d54-4090-b386-eed460d08440 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:00.163572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c59fc-8d54-4090-b386-eed460d08440 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:00.165242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-998c59fc-8d54-4090-b386-eed460d08440 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:00.239078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:00.253079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c59fc-8d54-4090-b386-eed460d08440 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:00.254670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c59fc-8d54-4090-b386-eed460d08440 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:00.272953 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 701/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:01.123765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47251950-b7a2-481b-8bc3-c35b6d094ab4 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:01.124334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47251950-b7a2-481b-8bc3-c35b6d094ab4 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:01.124334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47251950-b7a2-481b-8bc3-c35b6d094ab4 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:01.126252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 08:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:01.131041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc53586-6f1e-48de-b97f-d9256158cb2f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:01.131447 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc53586-6f1e-48de-b97f-d9256158cb2f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:01.131624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc53586-6f1e-48de-b97f-d9256158cb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:01.206432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-bdc53586-6f1e-48de-b97f-d9256158cb2f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:01.207622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-bdc53586-6f1e-48de-b97f-d9256158cb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:01.303515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 08:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:01.452791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47251950-b7a2-481b-8bc3-c35b6d094ab4 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:01.454320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-47251950-b7a2-481b-8bc3-c35b6d094ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:01.538421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:01.600744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47251950-b7a2-481b-8bc3-c35b6d094ab4 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:01.711211 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ad0e1-1cf3-4a20-b00f-9fc7b11a9559 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:01.711792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ad0e1-1cf3-4a20-b00f-9fc7b11a9559 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:01.712072 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ad0e1-1cf3-4a20-b00f-9fc7b11a9559 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:01.712346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ad0e1-1cf3-4a20-b00f-9fc7b11a9559 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:01.714008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d27ad0e1-1cf3-4a20-b00f-9fc7b11a9559 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:01.762170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:01.805597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ad0e1-1cf3-4a20-b00f-9fc7b11a9559 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:01.807284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ad0e1-1cf3-4a20-b00f-9fc7b11a9559 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:01.825122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 726/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:02.620089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e252a1-5727-4e1b-8e06-4a56c331c93d None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:02.620661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e252a1-5727-4e1b-8e06-4a56c331c93d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:02.620661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e252a1-5727-4e1b-8e06-4a56c331c93d None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:02.621985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 08:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:02.626848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca36468-15a5-4b1c-b1e6-b7c0d065231e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:02.627275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca36468-15a5-4b1c-b1e6-b7c0d065231e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:02.627413 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca36468-15a5-4b1c-b1e6-b7c0d065231e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:02.703012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-aca36468-15a5-4b1c-b1e6-b7c0d065231e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:02.704793 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-aca36468-15a5-4b1c-b1e6-b7c0d065231e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:02.808985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 08:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:02.958223 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e252a1-5727-4e1b-8e06-4a56c331c93d None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:02.959747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-04e252a1-5727-4e1b-8e06-4a56c331c93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:03.007333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:03.102772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e252a1-5727-4e1b-8e06-4a56c331c93d None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:03.216433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362a16d-0659-480a-86b4-c1effae84fc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:03.217041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362a16d-0659-480a-86b4-c1effae84fc8 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:03.217332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362a16d-0659-480a-86b4-c1effae84fc8 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:03.217609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362a16d-0659-480a-86b4-c1effae84fc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:03.219266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-6362a16d-0659-480a-86b4-c1effae84fc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:03.227939 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:03.310256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362a16d-0659-480a-86b4-c1effae84fc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:03.311887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362a16d-0659-480a-86b4-c1effae84fc8 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:03.330382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 704/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:04.129808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457eae-2df5-40d1-87b2-bd8cbb1b9836 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:04.130385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457eae-2df5-40d1-87b2-bd8cbb1b9836 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:04.130385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457eae-2df5-40d1-87b2-bd8cbb1b9836 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:04.131747 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 08:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:04.136547 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187967b0-95cc-44f0-ad0b-e20bc6046827 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:04.136989 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187967b0-95cc-44f0-ad0b-e20bc6046827 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:04.137139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187967b0-95cc-44f0-ad0b-e20bc6046827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:04.213590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-187967b0-95cc-44f0-ad0b-e20bc6046827 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:04.214797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-187967b0-95cc-44f0-ad0b-e20bc6046827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:04.282127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:04.312064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 08:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:04.462880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a457eae-2df5-40d1-87b2-bd8cbb1b9836 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:04.464438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8a457eae-2df5-40d1-87b2-bd8cbb1b9836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:04.482270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:04.609415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a457eae-2df5-40d1-87b2-bd8cbb1b9836 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:04.696997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:04.723105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b0bf0-e7bc-4615-bcd8-1ae2c989fba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:04.723698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b0bf0-e7bc-4615-bcd8-1ae2c989fba1 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:04.723988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b0bf0-e7bc-4615-bcd8-1ae2c989fba1 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:04.724266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b0bf0-e7bc-4615-bcd8-1ae2c989fba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:04.725994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d28b0bf0-e7bc-4615-bcd8-1ae2c989fba1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:04.819237 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b0bf0-e7bc-4615-bcd8-1ae2c989fba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:04.820868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b0bf0-e7bc-4615-bcd8-1ae2c989fba1 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:04.838952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 729/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:05.634698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64276514-92c7-40b3-88ee-49c1bf982d0f None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:05.635195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64276514-92c7-40b3-88ee-49c1bf982d0f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:05.635195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64276514-92c7-40b3-88ee-49c1bf982d0f None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:05.636653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 08:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:05.641873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad8309-b4da-4099-b39f-9642e4fed083 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:05.642251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad8309-b4da-4099-b39f-9642e4fed083 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:05.642435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad8309-b4da-4099-b39f-9642e4fed083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:05.721312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-30ad8309-b4da-4099-b39f-9642e4fed083 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:05.722530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-30ad8309-b4da-4099-b39f-9642e4fed083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:05.763937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:05.818091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 08:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:05.967528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64276514-92c7-40b3-88ee-49c1bf982d0f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:05.969010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-64276514-92c7-40b3-88ee-49c1bf982d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:06.113102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64276514-92c7-40b3-88ee-49c1bf982d0f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:06.167056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:06.225067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247718e-3bf1-4d49-9a7d-96a035890880 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:06.225641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247718e-3bf1-4d49-9a7d-96a035890880 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:06.225922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247718e-3bf1-4d49-9a7d-96a035890880 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:06.226194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247718e-3bf1-4d49-9a7d-96a035890880 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:06.227816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-9247718e-3bf1-4d49-9a7d-96a035890880 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:06.314200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247718e-3bf1-4d49-9a7d-96a035890880 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:06.315809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247718e-3bf1-4d49-9a7d-96a035890880 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:06.334415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 707/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:07.130579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f1e4c-4db6-472b-b1f1-24b4ede22dcc None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:07.131158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f1e4c-4db6-472b-b1f1-24b4ede22dcc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:07.131158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f1e4c-4db6-472b-b1f1-24b4ede22dcc None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:07.132518 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 08:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:07.137678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7fd4c-f285-4f2c-9c5e-0db352c37023 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:07.138079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7fd4c-f285-4f2c-9c5e-0db352c37023 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:07.138237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7fd4c-f285-4f2c-9c5e-0db352c37023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:07.214598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-e2f7fd4c-f285-4f2c-9c5e-0db352c37023 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:07.215798 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e2f7fd4c-f285-4f2c-9c5e-0db352c37023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:07.217832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:07.312023 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 08:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:39:07.462347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f1e4c-4db6-472b-b1f1-24b4ede22dcc None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:07.463977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-956f1e4c-4db6-472b-b1f1-24b4ede22dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:07.611732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f1e4c-4db6-472b-b1f1-24b4ede22dcc None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:07.639142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:07.725228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2596d5-abad-42dd-be87-017202f4e046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:07.725807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2596d5-abad-42dd-be87-017202f4e046 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:07.726091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2596d5-abad-42dd-be87-017202f4e046 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:07.726364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2596d5-abad-42dd-be87-017202f4e046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:07.728002 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1b2596d5-abad-42dd-be87-017202f4e046 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:07.817528 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2596d5-abad-42dd-be87-017202f4e046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:07.819121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2596d5-abad-42dd-be87-017202f4e046 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:07.836794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 732/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:09.732120 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf50ed7-fb71-4451-b7fb-d19f75cb8880 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:09.732691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf50ed7-fb71-4451-b7fb-d19f75cb8880 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:09.732804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf50ed7-fb71-4451-b7fb-d19f75cb8880 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:09.734143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:09.739480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeebb71-c210-45e2-85ab-e844db729fb2 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:09.739869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeebb71-c210-45e2-85ab-e844db729fb2 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:09.740046 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeebb71-c210-45e2-85ab-e844db729fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:09.781688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:09.806872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-1aeebb71-c210-45e2-85ab-e844db729fb2 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:09.808096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1aeebb71-c210-45e2-85ab-e844db729fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:09.906775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:09.945960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf50ed7-fb71-4451-b7fb-d19f75cb8880 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:09.947485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-acf50ed7-fb71-4451-b7fb-d19f75cb8880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:10.050999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf50ed7-fb71-4451-b7fb-d19f75cb8880 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:10.083620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:10.125932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef1409-5616-4f0b-8f1d-df2c071d367a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:10.126579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef1409-5616-4f0b-8f1d-df2c071d367a service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:10.126860 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef1409-5616-4f0b-8f1d-df2c071d367a service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:10.127133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef1409-5616-4f0b-8f1d-df2c071d367a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:10.128805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-79ef1409-5616-4f0b-8f1d-df2c071d367a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:10.234305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef1409-5616-4f0b-8f1d-df2c071d367a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:10.236268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef1409-5616-4f0b-8f1d-df2c071d367a service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:10.254383 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 710/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:12.383010 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f682bc9-4b19-4bfe-9528-50ec82e0a761 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:12.383502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f682bc9-4b19-4bfe-9528-50ec82e0a761 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:12.383601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f682bc9-4b19-4bfe-9528-50ec82e0a761 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:12.384992 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:12.390474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463b232-f2c1-4fc0-888d-651164f71464 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:12.391422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463b232-f2c1-4fc0-888d-651164f71464 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:12.391574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463b232-f2c1-4fc0-888d-651164f71464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:12.457336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-2463b232-f2c1-4fc0-888d-651164f71464 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:12.458535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2463b232-f2c1-4fc0-888d-651164f71464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:12.471485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:12.554284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:12.591260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f682bc9-4b19-4bfe-9528-50ec82e0a761 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:12.592848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9f682bc9-4b19-4bfe-9528-50ec82e0a761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:12.695876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f682bc9-4b19-4bfe-9528-50ec82e0a761 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:12.709350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:12.771928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5458f0f-958d-48fc-a74c-c8e0462eff14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:12.772602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5458f0f-958d-48fc-a74c-c8e0462eff14 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:12.772887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5458f0f-958d-48fc-a74c-c8e0462eff14 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:12.773165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5458f0f-958d-48fc-a74c-c8e0462eff14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:12.774814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e5458f0f-958d-48fc-a74c-c8e0462eff14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:12.883004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5458f0f-958d-48fc-a74c-c8e0462eff14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:12.884372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5458f0f-958d-48fc-a74c-c8e0462eff14 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:12.903779 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 735/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:15.191503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7550f-1799-45bf-a450-233653ab6bf1 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:15.192035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7550f-1799-45bf-a450-233653ab6bf1 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:15.192035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7550f-1799-45bf-a450-233653ab6bf1 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:15.193475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:15.198772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c44dfe-3d53-42aa-a6b2-1c72c917d0df None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:15.199209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c44dfe-3d53-42aa-a6b2-1c72c917d0df None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:15.199329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c44dfe-3d53-42aa-a6b2-1c72c917d0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:15.248097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:15.266412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c8c44dfe-3d53-42aa-a6b2-1c72c917d0df None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:15.267614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c8c44dfe-3d53-42aa-a6b2-1c72c917d0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:15.364163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:15.403230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7550f-1799-45bf-a450-233653ab6bf1 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:15.404812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c0c7550f-1799-45bf-a450-233653ab6bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:15.506208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7550f-1799-45bf-a450-233653ab6bf1 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:15.545277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:15.582103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3aa67-911e-4dd5-9512-838f8cecd3cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:15.582759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3aa67-911e-4dd5-9512-838f8cecd3cb service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:15.583045 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3aa67-911e-4dd5-9512-838f8cecd3cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:15.583322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3aa67-911e-4dd5-9512-838f8cecd3cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:15.585047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-15e3aa67-911e-4dd5-9512-838f8cecd3cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:15.692953 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3aa67-911e-4dd5-9512-838f8cecd3cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:15.694413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3aa67-911e-4dd5-9512-838f8cecd3cb service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:15.712478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 713/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:17.821712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e323d3-4325-45e4-b498-feb4942ab440 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:17.822252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e323d3-4325-45e4-b498-feb4942ab440 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:17.822252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e323d3-4325-45e4-b498-feb4942ab440 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:17.823662 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:17.829629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f0d8c-474c-4d2f-9710-b1f5444243d6 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:17.830058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f0d8c-474c-4d2f-9710-b1f5444243d6 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:17.830190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f0d8c-474c-4d2f-9710-b1f5444243d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:17.894908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-086f0d8c-474c-4d2f-9710-b1f5444243d6 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:17.896120 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-086f0d8c-474c-4d2f-9710-b1f5444243d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:17.916106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:17.991753 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:18.029961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e323d3-4325-45e4-b498-feb4942ab440 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:18.032008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-31e323d3-4325-45e4-b498-feb4942ab440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:18.135731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e323d3-4325-45e4-b498-feb4942ab440 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:18.155879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:18.211731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af9a02-5010-40ee-a87e-ce48ec0fa8c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:18.212386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af9a02-5010-40ee-a87e-ce48ec0fa8c6 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:18.212706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af9a02-5010-40ee-a87e-ce48ec0fa8c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:18.212992 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af9a02-5010-40ee-a87e-ce48ec0fa8c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:18.214651 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-39af9a02-5010-40ee-a87e-ce48ec0fa8c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:18.322734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af9a02-5010-40ee-a87e-ce48ec0fa8c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:18.324165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af9a02-5010-40ee-a87e-ce48ec0fa8c6 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:18.342824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 738/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:20.361286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4585aa-da77-4bec-b9f5-9582610f6130 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:20.361798 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4585aa-da77-4bec-b9f5-9582610f6130 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:20.361935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4585aa-da77-4bec-b9f5-9582610f6130 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:20.363246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:20.368579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620dfbb-a3c0-41ea-86c6-e71393f709dc None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:20.368990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620dfbb-a3c0-41ea-86c6-e71393f709dc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:20.369135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620dfbb-a3c0-41ea-86c6-e71393f709dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:20.433766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:20.437181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-3620dfbb-a3c0-41ea-86c6-e71393f709dc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:20.439291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3620dfbb-a3c0-41ea-86c6-e71393f709dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:20.532870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:20.570663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4585aa-da77-4bec-b9f5-9582610f6130 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:20.572168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8c4585aa-da77-4bec-b9f5-9582610f6130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:20.672642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4585aa-da77-4bec-b9f5-9582610f6130 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:20.720479 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:20.744974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc69c2e-70eb-4bb5-b44e-30191fde9f39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:20.745618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc69c2e-70eb-4bb5-b44e-30191fde9f39 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:20.745899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc69c2e-70eb-4bb5-b44e-30191fde9f39 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:20.746173 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc69c2e-70eb-4bb5-b44e-30191fde9f39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:20.747803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0cc69c2e-70eb-4bb5-b44e-30191fde9f39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:20.860130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc69c2e-70eb-4bb5-b44e-30191fde9f39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:20.861517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc69c2e-70eb-4bb5-b44e-30191fde9f39 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:20.879636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 716/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:23.022158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63c32f-13ea-4198-b4ca-6583453256fa None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:23.022694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63c32f-13ea-4198-b4ca-6583453256fa None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:23.022694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63c32f-13ea-4198-b4ca-6583453256fa None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:23.024148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:23.029438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f0322-d155-441a-9c23-0063a165e38e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:23.029850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f0322-d155-441a-9c23-0063a165e38e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:23.029993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f0322-d155-441a-9c23-0063a165e38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:23.097004 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-a76f0322-d155-441a-9c23-0063a165e38e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:23.098201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a76f0322-d155-441a-9c23-0063a165e38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:23.127020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:23.194311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:23.233050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63c32f-13ea-4198-b4ca-6583453256fa None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:23.235221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-be63c32f-13ea-4198-b4ca-6583453256fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:23.339151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63c32f-13ea-4198-b4ca-6583453256fa None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:23.367664 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:23.415320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d50ff-cd70-455f-ac3b-3d9039444f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:23.415983 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d50ff-cd70-455f-ac3b-3d9039444f2c service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:23.416273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d50ff-cd70-455f-ac3b-3d9039444f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:23.416555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d50ff-cd70-455f-ac3b-3d9039444f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:23.418242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-265d50ff-cd70-455f-ac3b-3d9039444f2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:23.526093 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d50ff-cd70-455f-ac3b-3d9039444f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:23.527528 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d50ff-cd70-455f-ac3b-3d9039444f2c service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:23.546314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 741/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:25.561518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bfdc9-f656-4367-a38a-c838c1374c88 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:25.562009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bfdc9-f656-4367-a38a-c838c1374c88 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:25.562009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bfdc9-f656-4367-a38a-c838c1374c88 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:25.563904 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:39:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:25.569283 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3e514-4669-4609-947e-77cf2812e30d None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:25.569710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3e514-4669-4609-947e-77cf2812e30d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:25.569848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3e514-4669-4609-947e-77cf2812e30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:25.636716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-48c3e514-4669-4609-947e-77cf2812e30d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:25.638173 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-48c3e514-4669-4609-947e-77cf2812e30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:25.646437 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:25.733126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:25.770975 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bfdc9-f656-4367-a38a-c838c1374c88 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:25.772531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-462bfdc9-f656-4367-a38a-c838c1374c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:25.873460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bfdc9-f656-4367-a38a-c838c1374c88 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:25.927739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:25.946102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105a8d9-939c-48a2-8bd6-89ef1d1cebd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:25.946745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105a8d9-939c-48a2-8bd6-89ef1d1cebd9 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:25.947027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105a8d9-939c-48a2-8bd6-89ef1d1cebd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:25.947300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105a8d9-939c-48a2-8bd6-89ef1d1cebd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:25.948957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c105a8d9-939c-48a2-8bd6-89ef1d1cebd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:26.054636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105a8d9-939c-48a2-8bd6-89ef1d1cebd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:26.056064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105a8d9-939c-48a2-8bd6-89ef1d1cebd9 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:26.075503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 719/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:28.104265 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b2883-c144-405b-be3b-3a99e9160254 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:28.104897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b2883-c144-405b-be3b-3a99e9160254 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:28.104897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b2883-c144-405b-be3b-3a99e9160254 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:28.106294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:28.111439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69e208-9d94-4963-a996-d814b120bded None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:28.111895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69e208-9d94-4963-a996-d814b120bded None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:28.112008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69e208-9d94-4963-a996-d814b120bded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:28.177533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ef69e208-9d94-4963-a996-d814b120bded None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:28.178754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ef69e208-9d94-4963-a996-d814b120bded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:28.217255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:28.274501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:28.310827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b2883-c144-405b-be3b-3a99e9160254 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:28.313011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d23b2883-c144-405b-be3b-3a99e9160254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:28.416696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b2883-c144-405b-be3b-3a99e9160254 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:28.455040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:28.490471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b7e6f-8061-4e24-b054-261a35fd36df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:28.491117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b7e6f-8061-4e24-b054-261a35fd36df service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:28.491401 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b7e6f-8061-4e24-b054-261a35fd36df service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:28.491680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b7e6f-8061-4e24-b054-261a35fd36df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:28.493387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9a6b7e6f-8061-4e24-b054-261a35fd36df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:28.600798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b7e6f-8061-4e24-b054-261a35fd36df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:28.602690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b7e6f-8061-4e24-b054-261a35fd36df service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:28.621965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 744/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:30.612077 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2d88a-e23d-4197-975b-188a5e7b2844 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:30.612660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2d88a-e23d-4197-975b-188a5e7b2844 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:30.612660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2d88a-e23d-4197-975b-188a5e7b2844 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:30.613959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:30.619679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b72f54-1957-414a-99c8-0aac610973c7 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:30.620081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b72f54-1957-414a-99c8-0aac610973c7 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:30.620245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b72f54-1957-414a-99c8-0aac610973c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:30.685310 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-41b72f54-1957-414a-99c8-0aac610973c7 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:30.686520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-41b72f54-1957-414a-99c8-0aac610973c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:30.704193 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:30.783382 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:30.820092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2d88a-e23d-4197-975b-188a5e7b2844 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:30.821690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c7f2d88a-e23d-4197-975b-188a5e7b2844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:30.925326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2d88a-e23d-4197-975b-188a5e7b2844 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:30.990460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:30.997728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cec166-6432-4704-b50f-a30d6d162d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:30.998380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cec166-6432-4704-b50f-a30d6d162d0e service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:30.998669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cec166-6432-4704-b50f-a30d6d162d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:30.998946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cec166-6432-4704-b50f-a30d6d162d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:31.000658 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-82cec166-6432-4704-b50f-a30d6d162d0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:31.107331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cec166-6432-4704-b50f-a30d6d162d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:31.108788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cec166-6432-4704-b50f-a30d6d162d0e service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:31.126867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 722/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:33.127007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51f962-370d-4785-9464-d60b541c4142 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:33.127604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51f962-370d-4785-9464-d60b541c4142 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:33.127604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51f962-370d-4785-9464-d60b541c4142 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:33.128981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:33.134288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af9492-3372-466e-a761-7e7302aeffc1 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:33.134735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af9492-3372-466e-a761-7e7302aeffc1 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:33.134881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af9492-3372-466e-a761-7e7302aeffc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:33.201636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-f1af9492-3372-466e-a761-7e7302aeffc1 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:33.202913 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f1af9492-3372-466e-a761-7e7302aeffc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:33.250745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:33.299827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:33.337119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51f962-370d-4785-9464-d60b541c4142 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:33.338701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ec51f962-370d-4785-9464-d60b541c4142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:33.443079 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51f962-370d-4785-9464-d60b541c4142 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:33.491107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:33.516968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60577ef0-5ebe-48b0-8dc9-83df9cd5cec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:33.517617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60577ef0-5ebe-48b0-8dc9-83df9cd5cec8 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:33.517901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60577ef0-5ebe-48b0-8dc9-83df9cd5cec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:33.518177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60577ef0-5ebe-48b0-8dc9-83df9cd5cec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:33.519816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-60577ef0-5ebe-48b0-8dc9-83df9cd5cec8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:33.626455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60577ef0-5ebe-48b0-8dc9-83df9cd5cec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:33.627873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60577ef0-5ebe-48b0-8dc9-83df9cd5cec8 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:33.646209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 747/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:43.297992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=22 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:43.643816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=66 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:53.026211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53519cfa-8477-4ffc-9cbf-1246b48fe592 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:53.026723 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53519cfa-8477-4ffc-9cbf-1246b48fe592 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:53.026723 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53519cfa-8477-4ffc-9cbf-1246b48fe592 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:53.028115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 724/1471] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:53.033780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1c87d-6ec0-487e-9a12-25bd9cab5701 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:53.034208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1c87d-6ec0-487e-9a12-25bd9cab5701 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:53.034343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1c87d-6ec0-487e-9a12-25bd9cab5701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:53.099754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-64d1c87d-6ec0-487e-9a12-25bd9cab5701 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:53.100999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-64d1c87d-6ec0-487e-9a12-25bd9cab5701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:53.198950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 748/1472] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:53.280192 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1645b-6c0a-4144-a31c-bab77017450f None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:53.280427 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1645b-6c0a-4144-a31c-bab77017450f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:53.280750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1645b-6c0a-4144-a31c-bab77017450f None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:53.282638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 725/1473] 10.222.0.52 () {54 vars in 966 bytes} [Mon Jul 25 08:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:53.287465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe87ca6-6666-4ce4-ba02-e1af6c3d3748 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:53.287758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe87ca6-6666-4ce4-ba02-e1af6c3d3748 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:53.288068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe87ca6-6666-4ce4-ba02-e1af6c3d3748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:53.363530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-3fe87ca6-6666-4ce4-ba02-e1af6c3d3748 None None] MFA Rules not processed for user `925bf2da4df74e86aa84ec62a267d088`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:53.364765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3fe87ca6-6666-4ce4-ba02-e1af6c3d3748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:53.384747 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:53.441607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 749/1474] 10.222.0.52 () {58 vars in 1063 bytes} [Mon Jul 25 08:39:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:53.449212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1645b-6c0a-4144-a31c-bab77017450f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:53.450715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-13e1645b-6c0a-4144-a31c-bab77017450f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:53.551496 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1645b-6c0a-4144-a31c-bab77017450f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:53.621745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa9f82-5ac5-4a72-a4af-a85a5ef0abd4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '925bf2da4df74e86aa84ec62a267d088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:53.622386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa9f82-5ac5-4a72-a4af-a85a5ef0abd4 service glance] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:53.622668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa9f82-5ac5-4a72-a4af-a85a5ef0abd4 service glance] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:53.622943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa9f82-5ac5-4a72-a4af-a85a5ef0abd4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:53.624553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1cfa9f82-5ac5-4a72-a4af-a85a5ef0abd4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:53.678771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:53.733022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa9f82-5ac5-4a72-a4af-a85a5ef0abd4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:53.734447 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa9f82-5ac5-4a72-a4af-a85a5ef0abd4 service glance] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:53.753361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 726/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:39:55.826813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25c4b3-172e-4e85-a03f-43975c286a92 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:55.827388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25c4b3-172e-4e85-a03f-43975c286a92 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:55.827388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25c4b3-172e-4e85-a03f-43975c286a92 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:55.829265 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 750/1476] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:39:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:55.834456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c90c5c-ac86-41f5-bff5-8f5bfcd4fbc7 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:55.834868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c90c5c-ac86-41f5-bff5-8f5bfcd4fbc7 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:55.835010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c90c5c-ac86-41f5-bff5-8f5bfcd4fbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:55.901368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-44c90c5c-ac86-41f5-bff5-8f5bfcd4fbc7 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:55.902561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-44c90c5c-ac86-41f5-bff5-8f5bfcd4fbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:55.997577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 727/1477] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:56.021275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25c4b3-172e-4e85-a03f-43975c286a92 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:56.022786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fa25c4b3-172e-4e85-a03f-43975c286a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:56.146228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25c4b3-172e-4e85-a03f-43975c286a92 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:56.193919 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:56.237584 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060aa67-0c17-47fd-a3e0-9d4e0f39e478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:56.238240 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060aa67-0c17-47fd-a3e0-9d4e0f39e478 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:56.238519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060aa67-0c17-47fd-a3e0-9d4e0f39e478 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:56.238793 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060aa67-0c17-47fd-a3e0-9d4e0f39e478 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:56.239517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060aa67-0c17-47fd-a3e0-9d4e0f39e478 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:56.241242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: WARNING py.warnings [None req-3060aa67-0c17-47fd-a3e0-9d4e0f39e478 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:39:56.241242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: warnings.warn(msg) Jul 25 08:39:56.241242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: Jul 25 08:39:56.242376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060aa67-0c17-47fd-a3e0-9d4e0f39e478 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:56.261119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 751/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Jul 25 08:39:56 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:39:58.250260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff7af4-547c-4dc9-b966-bf6677345c9e None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:58.250820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff7af4-547c-4dc9-b966-bf6677345c9e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:58.250820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff7af4-547c-4dc9-b966-bf6677345c9e None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:58.252216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 728/1479] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:39:58.257410 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034b0cb-bca4-45b7-a6d1-fec065184cc8 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:58.257806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034b0cb-bca4-45b7-a6d1-fec065184cc8 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:58.257976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034b0cb-bca4-45b7-a6d1-fec065184cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:58.325954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a034b0cb-bca4-45b7-a6d1-fec065184cc8 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:39:58.327160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a034b0cb-bca4-45b7-a6d1-fec065184cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:58.423701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 752/1480] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:39:58.465275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff7af4-547c-4dc9-b966-bf6677345c9e None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:39:58.466857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a8ff7af4-547c-4dc9-b966-bf6677345c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:58.477111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:58.609896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff7af4-547c-4dc9-b966-bf6677345c9e None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:39:58.687887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:39:58.720818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306f3ea-69f7-4f94-8a81-87d29cdb2061 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:39:58.721467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306f3ea-69f7-4f94-8a81-87d29cdb2061 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:39:58.721750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306f3ea-69f7-4f94-8a81-87d29cdb2061 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:39:58.722025 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306f3ea-69f7-4f94-8a81-87d29cdb2061 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:39:58.723672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c306f3ea-69f7-4f94-8a81-87d29cdb2061 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:39:58.810551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306f3ea-69f7-4f94-8a81-87d29cdb2061 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:39:58.812128 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306f3ea-69f7-4f94-8a81-87d29cdb2061 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:39:58.830434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 729/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:39:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:40:00.989493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424476f3-51b8-49e5-8360-be299c8a2848 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:40:00.990100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424476f3-51b8-49e5-8360-be299c8a2848 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:40:00.990100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424476f3-51b8-49e5-8360-be299c8a2848 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:40:00.992187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 753/1482] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:40:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:40:00.998036 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd003e9-9388-4dd2-9dd0-05366df00bfa None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:40:00.998503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd003e9-9388-4dd2-9dd0-05366df00bfa None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:40:00.998609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd003e9-9388-4dd2-9dd0-05366df00bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:40:01.064072 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-4dd003e9-9388-4dd2-9dd0-05366df00bfa None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:40:01.065306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4dd003e9-9388-4dd2-9dd0-05366df00bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:40:01.113400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:40:01.163716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 730/1483] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:40:01.199572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424476f3-51b8-49e5-8360-be299c8a2848 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:40:01.201156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-424476f3-51b8-49e5-8360-be299c8a2848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:40:01.225884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:40:01.305333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424476f3-51b8-49e5-8360-be299c8a2848 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:40:01.378315 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fd3d2-57d2-4eb3-b520-7e8246f4d60f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:40:01.378961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fd3d2-57d2-4eb3-b520-7e8246f4d60f service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:40:01.379244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fd3d2-57d2-4eb3-b520-7e8246f4d60f service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:40:01.379522 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fd3d2-57d2-4eb3-b520-7e8246f4d60f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:40:01.381198 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-445fd3d2-57d2-4eb3-b520-7e8246f4d60f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:40:01.455185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:40:01.490085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fd3d2-57d2-4eb3-b520-7e8246f4d60f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:40:01.491528 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fd3d2-57d2-4eb3-b520-7e8246f4d60f service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:40:01.510532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 754/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:40:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:40:03.547493 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ef329-fb7b-4100-be1c-0d97d72ff34d None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:40:03.548086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ef329-fb7b-4100-be1c-0d97d72ff34d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:40:03.548086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ef329-fb7b-4100-be1c-0d97d72ff34d None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:40:03.549462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 731/1485] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:40:03.554759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a91f4-4c32-4b39-b5a8-087af4c74fd7 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:40:03.555201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a91f4-4c32-4b39-b5a8-087af4c74fd7 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:40:03.555330 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a91f4-4c32-4b39-b5a8-087af4c74fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:40:03.620903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-b08a91f4-4c32-4b39-b5a8-087af4c74fd7 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:40:03.622038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b08a91f4-4c32-4b39-b5a8-087af4c74fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:40:03.720531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 755/1486] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:40:03.758466 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ef329-fb7b-4100-be1c-0d97d72ff34d None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:40:03.760007 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e44ef329-fb7b-4100-be1c-0d97d72ff34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:40:03.862912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ef329-fb7b-4100-be1c-0d97d72ff34d None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:40:03.933096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:40:03.935071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca7e30-1f67-4582-b674-8b3744bb2ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:40:03.935794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca7e30-1f67-4582-b674-8b3744bb2ece service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:40:03.936306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca7e30-1f67-4582-b674-8b3744bb2ece service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:40:03.936791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca7e30-1f67-4582-b674-8b3744bb2ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:40:03.940034 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-caca7e30-1f67-4582-b674-8b3744bb2ece service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:40:04.048151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca7e30-1f67-4582-b674-8b3744bb2ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:40:04.049628 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca7e30-1f67-4582-b674-8b3744bb2ece service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:40:04.069348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 732/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:40:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:40:06.120888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616aff58-f5df-4c9d-b1bd-bf4d43fecd1b None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:40:06.121490 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616aff58-f5df-4c9d-b1bd-bf4d43fecd1b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:40:06.121490 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616aff58-f5df-4c9d-b1bd-bf4d43fecd1b None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:40:06.122914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 756/1488] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:40:06.128011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97980b71-267c-4e59-882d-1706a4f51c1b None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:40:06.128398 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97980b71-267c-4e59-882d-1706a4f51c1b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:40:06.128613 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97980b71-267c-4e59-882d-1706a4f51c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:40:06.193083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-97980b71-267c-4e59-882d-1706a4f51c1b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:40:06.194277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-97980b71-267c-4e59-882d-1706a4f51c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:40:06.245727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:40:06.291020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 733/1489] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:40:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:40:06.331470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616aff58-f5df-4c9d-b1bd-bf4d43fecd1b None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:40:06.333110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-616aff58-f5df-4c9d-b1bd-bf4d43fecd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:40:06.369382 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:40:06.437109 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616aff58-f5df-4c9d-b1bd-bf4d43fecd1b None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:40:06.508855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e94c32-0731-41f1-8e2b-dfa1cc097e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:40:06.509500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e94c32-0731-41f1-8e2b-dfa1cc097e04 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:40:06.509784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e94c32-0731-41f1-8e2b-dfa1cc097e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:40:06.510064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e94c32-0731-41f1-8e2b-dfa1cc097e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:40:06.511697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-46e94c32-0731-41f1-8e2b-dfa1cc097e04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:40:06.590124 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:40:06.619613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e94c32-0731-41f1-8e2b-dfa1cc097e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:40:06.621090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e94c32-0731-41f1-8e2b-dfa1cc097e04 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:40:06.639505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 757/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:40:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:40:08.799169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed76d4-b0a2-4f46-87cb-f5429b63787a None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:40:08.799709 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed76d4-b0a2-4f46-87cb-f5429b63787a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:40:08.799824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed76d4-b0a2-4f46-87cb-f5429b63787a None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:40:08.801229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 734/1491] 10.222.0.52 () {54 vars in 926 bytes} [Mon Jul 25 08:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:40:08.806495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a8e65-76f9-49b3-b6d6-dac91c9a12b0 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:40:08.806931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a8e65-76f9-49b3-b6d6-dac91c9a12b0 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:40:08.807059 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a8e65-76f9-49b3-b6d6-dac91c9a12b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:40:08.872754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-681a8e65-76f9-49b3-b6d6-dac91c9a12b0 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:40:08.873968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-681a8e65-76f9-49b3-b6d6-dac91c9a12b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:40:08.971195 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 758/1492] 10.222.0.52 () {58 vars in 1023 bytes} [Mon Jul 25 08:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:40:09.010279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed76d4-b0a2-4f46-87cb-f5429b63787a None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:40:09.011834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-07ed76d4-b0a2-4f46-87cb-f5429b63787a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:40:09.115997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed76d4-b0a2-4f46-87cb-f5429b63787a None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:40:09.188565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebd94d-e760-4fa5-8501-a7ec7da025ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:40:09.189235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebd94d-e760-4fa5-8501-a7ec7da025ec service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:40:09.189520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebd94d-e760-4fa5-8501-a7ec7da025ec service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:40:09.189794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebd94d-e760-4fa5-8501-a7ec7da025ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:40:09.191453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-14ebd94d-e760-4fa5-8501-a7ec7da025ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:40:09.200356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:40:09.299379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebd94d-e760-4fa5-8501-a7ec7da025ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:40:09.300877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebd94d-e760-4fa5-8501-a7ec7da025ec service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:40:09.320204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 735/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:40:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:40:18.969365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=82 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:40:19.317899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=50 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:41:56.080691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a491a-4862-4db1-b7e6-1db4ee02731b None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:56.081312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a491a-4862-4db1-b7e6-1db4ee02731b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:56.081312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a491a-4862-4db1-b7e6-1db4ee02731b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:56.156707 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a47a491a-4862-4db1-b7e6-1db4ee02731b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:41:56.158062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a47a491a-4862-4db1-b7e6-1db4ee02731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:56.254320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 759/1494] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:41:56.262306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c6da6-5510-4b02-b0b9-6c27ed42125d None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:56.262813 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c6da6-5510-4b02-b0b9-6c27ed42125d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:56.262919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c6da6-5510-4b02-b0b9-6c27ed42125d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:56.338155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-7b7c6da6-5510-4b02-b0b9-6c27ed42125d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:41:56.339284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7b7c6da6-5510-4b02-b0b9-6c27ed42125d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:56.434079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 736/1495] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:41:56.439254 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a491a-4862-4db1-b7e6-1db4ee02731b None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:41:56.440475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a47a491a-4862-4db1-b7e6-1db4ee02731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:56.451945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a491a-4862-4db1-b7e6-1db4ee02731b None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:41:56.514472 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:41:56.548932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214cb43-6c74-427e-a66a-21164a8e600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:41:56.549595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214cb43-6c74-427e-a66a-21164a8e600c admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:56.549872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214cb43-6c74-427e-a66a-21164a8e600c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:56.550142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214cb43-6c74-427e-a66a-21164a8e600c admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:56.550860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214cb43-6c74-427e-a66a-21164a8e600c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:41:56.551163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214cb43-6c74-427e-a66a-21164a8e600c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:41:56.552180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: WARNING py.warnings [None req-e214cb43-6c74-427e-a66a-21164a8e600c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:41:56.552180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: warnings.warn(msg) Jul 25 08:41:56.552180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: Jul 25 08:41:56.553088 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214cb43-6c74-427e-a66a-21164a8e600c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:41:56.564939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 760/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:41:56.573054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c6da6-5510-4b02-b0b9-6c27ed42125d None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:41:56.582239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c6da6-5510-4b02-b0b9-6c27ed42125d None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:41:56.644737 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:41:56.678214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c51ea-d69a-402f-b62b-a3ea962aa859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:41:56.678876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c51ea-d69a-402f-b62b-a3ea962aa859 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:56.679160 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c51ea-d69a-402f-b62b-a3ea962aa859 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:56.679438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c51ea-d69a-402f-b62b-a3ea962aa859 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:56.680141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c51ea-d69a-402f-b62b-a3ea962aa859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:41:56.681885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: WARNING py.warnings [None req-240c51ea-d69a-402f-b62b-a3ea962aa859 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:41:56.681885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: warnings.warn(msg) Jul 25 08:41:56.681885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: Jul 25 08:41:56.683078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c51ea-d69a-402f-b62b-a3ea962aa859 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:41:56.738633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 737/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:41:56 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:41:56.743291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214cb43-6c74-427e-a66a-21164a8e600c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:41:56.744893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e214cb43-6c74-427e-a66a-21164a8e600c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:56.777435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214cb43-6c74-427e-a66a-21164a8e600c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:41:56.870471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288dd93-9432-4b7d-8b14-d96616aecc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:41:56.871141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288dd93-9432-4b7d-8b14-d96616aecc69 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:56.871426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288dd93-9432-4b7d-8b14-d96616aecc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:56.871697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288dd93-9432-4b7d-8b14-d96616aecc69 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:56.872394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288dd93-9432-4b7d-8b14-d96616aecc69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:41:56.873441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: WARNING py.warnings [None req-0288dd93-9432-4b7d-8b14-d96616aecc69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:41:56.873441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: warnings.warn(msg) Jul 25 08:41:56.873441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: Jul 25 08:41:56.874678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288dd93-9432-4b7d-8b14-d96616aecc69 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:41:56.915891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 761/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:41:56 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:41:56.920106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c51ea-d69a-402f-b62b-a3ea962aa859 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:41:56.922355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-240c51ea-d69a-402f-b62b-a3ea962aa859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:56.983018 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c51ea-d69a-402f-b62b-a3ea962aa859 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:41:57.066331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:41:57.078613 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312396c7-cbc5-4ac0-81dc-6959b8b3d849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:41:57.079277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312396c7-cbc5-4ac0-81dc-6959b8b3d849 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:57.079555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312396c7-cbc5-4ac0-81dc-6959b8b3d849 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:57.079828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312396c7-cbc5-4ac0-81dc-6959b8b3d849 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:57.080549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312396c7-cbc5-4ac0-81dc-6959b8b3d849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:41:57.081585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: WARNING py.warnings [None req-312396c7-cbc5-4ac0-81dc-6959b8b3d849 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:41:57.081585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: warnings.warn(msg) Jul 25 08:41:57.081585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: Jul 25 08:41:57.082466 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312396c7-cbc5-4ac0-81dc-6959b8b3d849 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:41:57.095201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 738/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:41:57.099256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288dd93-9432-4b7d-8b14-d96616aecc69 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:41:57.100805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0288dd93-9432-4b7d-8b14-d96616aecc69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:57.112301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288dd93-9432-4b7d-8b14-d96616aecc69 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:41:57.136952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:41:57.209942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949be18e-3d2b-4da5-9a28-31bc9c3ba00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:41:57.210545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949be18e-3d2b-4da5-9a28-31bc9c3ba00f admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:57.210825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949be18e-3d2b-4da5-9a28-31bc9c3ba00f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:57.211097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949be18e-3d2b-4da5-9a28-31bc9c3ba00f admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:57.211824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949be18e-3d2b-4da5-9a28-31bc9c3ba00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:41:57.212864 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: WARNING py.warnings [None req-949be18e-3d2b-4da5-9a28-31bc9c3ba00f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:41:57.212864 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: warnings.warn(msg) Jul 25 08:41:57.212864 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: Jul 25 08:41:57.213746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949be18e-3d2b-4da5-9a28-31bc9c3ba00f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:41:57.226586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 762/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:41:57.230749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312396c7-cbc5-4ac0-81dc-6959b8b3d849 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:41:57.232265 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-312396c7-cbc5-4ac0-81dc-6959b8b3d849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:57.264937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312396c7-cbc5-4ac0-81dc-6959b8b3d849 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:41:57.359016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f063a-a955-43c5-8b72-e972d959ce55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:41:57.359911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f063a-a955-43c5-8b72-e972d959ce55 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:57.360196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f063a-a955-43c5-8b72-e972d959ce55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:57.360474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f063a-a955-43c5-8b72-e972d959ce55 admin admin] PATH_INFO: `/v3/projects/3a64cc9ec854437f958caa693bfdcb63/users/069a8d6bdc0045a2af6f4b5a4dac8b1d/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:57.382179 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f063a-a955-43c5-8b72-e972d959ce55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a64cc9ec854437f958caa693bfdcb63, user_id=069a8d6bdc0045a2af6f4b5a4dac8b1d, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:41:57.383210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: WARNING py.warnings [None req-b48f063a-a955-43c5-8b72-e972d959ce55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:41:57.383210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: warnings.warn(msg) Jul 25 08:41:57.383210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: Jul 25 08:41:57.385196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f063a-a955-43c5-8b72-e972d959ce55 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:41:57.410252 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 739/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:41:57 2022] PUT /identity/v3/projects/3a64cc9ec854437f958caa693bfdcb63/users/069a8d6bdc0045a2af6f4b5a4dac8b1d/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:41:57.422731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c180eb-2fb7-4dc3-a595-d9de9338e615 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:57.423044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c180eb-2fb7-4dc3-a595-d9de9338e615 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:57.423327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c180eb-2fb7-4dc3-a595-d9de9338e615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:57.480296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c5c180eb-2fb7-4dc3-a595-d9de9338e615 None None] MFA Rules not processed for user `069a8d6bdc0045a2af6f4b5a4dac8b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:41:57.481485 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c5c180eb-2fb7-4dc3-a595-d9de9338e615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:57.558566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 763/1502] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:41:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:41:57.572382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf81098-c1c6-45c6-8651-0e09bb3349da None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:57.572701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf81098-c1c6-45c6-8651-0e09bb3349da None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:57.573004 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf81098-c1c6-45c6-8651-0e09bb3349da None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:57.574388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 740/1503] 10.222.0.52 () {54 vars in 964 bytes} [Mon Jul 25 08:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:41:57.579595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc42917-3293-408b-8798-1027dabb0ffc None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:57.579886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc42917-3293-408b-8798-1027dabb0ffc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:57.580168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc42917-3293-408b-8798-1027dabb0ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:57.610761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:41:57.656666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-bdc42917-3293-408b-8798-1027dabb0ffc None None] MFA Rules not processed for user `84871b711bcb47efb039ce40766cf3db`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:41:57.657888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-bdc42917-3293-408b-8798-1027dabb0ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:57.732196 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 764/1504] 10.222.0.52 () {58 vars in 1061 bytes} [Mon Jul 25 08:41:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:41:57.739802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf81098-c1c6-45c6-8651-0e09bb3349da None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:41:57.741370 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-eaf81098-c1c6-45c6-8651-0e09bb3349da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:57.792292 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:41:57.847593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf81098-c1c6-45c6-8651-0e09bb3349da None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:41:57.920203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2466d1d-0185-45bc-bcc8-f11d685a40bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:41:57.920928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2466d1d-0185-45bc-bcc8-f11d685a40bd service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:57.921718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2466d1d-0185-45bc-bcc8-f11d685a40bd service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:57.921999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2466d1d-0185-45bc-bcc8-f11d685a40bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:57.923653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e2466d1d-0185-45bc-bcc8-f11d685a40bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:58.019557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2466d1d-0185-45bc-bcc8-f11d685a40bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:41:58.020957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2466d1d-0185-45bc-bcc8-f11d685a40bd service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:41:58.024451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:41:58.039366 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 741/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:41:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:41:58.085222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b027f0-3674-4b20-a116-6655e2a65611 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:58.085515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b027f0-3674-4b20-a116-6655e2a65611 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:58.085795 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b027f0-3674-4b20-a116-6655e2a65611 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:58.087097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 765/1506] 10.222.0.52 () {52 vars in 830 bytes} [Mon Jul 25 08:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:41:58.090620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2466d1d-0185-45bc-bcc8-f11d685a40bd service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:41:58.091830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e2466d1d-0185-45bc-bcc8-f11d685a40bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:58.190552 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2466d1d-0185-45bc-bcc8-f11d685a40bd service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:41:58.284757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:41:58.285593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:58.285877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:58.286152 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] PATH_INFO: `/v3/users/069a8d6bdc0045a2af6f4b5a4dac8b1d` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:58.299202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=069a8d6bdc0045a2af6f4b5a4dac8b1d)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:41:58.300255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: WARNING py.warnings [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:41:58.300255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: warnings.warn(msg) Jul 25 08:41:58.300255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: Jul 25 08:41:58.301515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:41:58.311349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:41:58.365236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '069a8d6bdc0045a2af6f4b5a4dac8b1d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:41:58.375719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '069a8d6bdc0045a2af6f4b5a4dac8b1d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:41:58.393747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '069a8d6bdc0045a2af6f4b5a4dac8b1d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:41:58.406298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '069a8d6bdc0045a2af6f4b5a4dac8b1d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:41:58.406830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '069a8d6bdc0045a2af6f4b5a4dac8b1d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:41:58.412670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 742/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:41:58 2022] DELETE /identity/v3/users/069a8d6bdc0045a2af6f4b5a4dac8b1d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:41:58.417168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6b8dd-f5f1-4ca6-a565-66ff87966ec8 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:58.417621 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6b8dd-f5f1-4ca6-a565-66ff87966ec8 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:58.417729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6b8dd-f5f1-4ca6-a565-66ff87966ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:58.475720 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-98d6b8dd-f5f1-4ca6-a565-66ff87966ec8 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:41:58.476963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-98d6b8dd-f5f1-4ca6-a565-66ff87966ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:58.552308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:41:58.565427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 766/1508] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:41:58.574888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:41:58.576152 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:58.676037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a3d38-5a27-40d0-985a-c37e9a7f0593 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:41:58.730379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:41:58.751330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6cd5a-8e97-4c0f-b1cf-3546b007f24e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:41:58.751981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6cd5a-8e97-4c0f-b1cf-3546b007f24e service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:58.752267 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6cd5a-8e97-4c0f-b1cf-3546b007f24e service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:58.752544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6cd5a-8e97-4c0f-b1cf-3546b007f24e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:58.754207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-09a6cd5a-8e97-4c0f-b1cf-3546b007f24e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:58.861973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6cd5a-8e97-4c0f-b1cf-3546b007f24e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:41:58.863339 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6cd5a-8e97-4c0f-b1cf-3546b007f24e service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:41:58.881599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 743/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:41:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:41:59.227401 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6b8dd-f5f1-4ca6-a565-66ff87966ec8 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:41:59.228837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-98d6b8dd-f5f1-4ca6-a565-66ff87966ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:41:59.240721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6b8dd-f5f1-4ca6-a565-66ff87966ec8 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:41:59.343084 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:41:59.343915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:41:59.344199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:41:59.344470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] PATH_INFO: `/v3/projects/3a64cc9ec854437f958caa693bfdcb63` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:41:59.354342 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a64cc9ec854437f958caa693bfdcb63)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:41:59.355376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: WARNING py.warnings [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:41:59.355376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: warnings.warn(msg) Jul 25 08:41:59.355376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: Jul 25 08:41:59.356560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:41:59.413616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a64cc9ec854437f958caa693bfdcb63', 'request_id': 'req-b55b0ffe-573f-4806-b852-d79acaf47f7f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:41:59.423597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a64cc9ec854437f958caa693bfdcb63', 'request_id': 'req-b55b0ffe-573f-4806-b852-d79acaf47f7f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:41:59.429782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a64cc9ec854437f958caa693bfdcb63', 'request_id': 'req-b55b0ffe-573f-4806-b852-d79acaf47f7f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:41:59.459302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] The token cache is being invalidate because project 3a64cc9ec854437f958caa693bfdcb63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:41:59.459589 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:41:59.460139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b55b0ffe-573f-4806-b852-d79acaf47f7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:41:59.461525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 767/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:41:59 2022] DELETE /identity/v3/projects/3a64cc9ec854437f958caa693bfdcb63 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:42:08.879261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=62 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:42:09.457362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:43:41.088473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5d674-0632-4b89-a5c8-699b49b43d86 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:41.089102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5d674-0632-4b89-a5c8-699b49b43d86 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:41.089102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5d674-0632-4b89-a5c8-699b49b43d86 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:41.090447 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 08:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:43:41.095246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0e5de-812f-45db-b940-8e1f2f3868be None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:41.095699 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0e5de-812f-45db-b940-8e1f2f3868be None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:41.095821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0e5de-812f-45db-b940-8e1f2f3868be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:41.177108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-57f0e5de-812f-45db-b940-8e1f2f3868be None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:43:41.178321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-57f0e5de-812f-45db-b940-8e1f2f3868be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:43:41.282730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 08:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:43:41.445470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5d674-0632-4b89-a5c8-699b49b43d86 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:43:41.447598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-bff5d674-0632-4b89-a5c8-699b49b43d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:43:41.565782 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5d674-0632-4b89-a5c8-699b49b43d86 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:43:41.571534 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a981fa5-29c9-435e-87b9-5a4ea56b7179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:43:41.572215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a981fa5-29c9-435e-87b9-5a4ea56b7179 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:41.572498 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a981fa5-29c9-435e-87b9-5a4ea56b7179 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:41.572805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a981fa5-29c9-435e-87b9-5a4ea56b7179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:41.580853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a981fa5-29c9-435e-87b9-5a4ea56b7179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:43:41.582419 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a981fa5-29c9-435e-87b9-5a4ea56b7179 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:43:41.591658 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 745/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:43:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:43:41.749769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3188ca7-76b6-487e-bba0-3ede13ff5411 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:41.750108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3188ca7-76b6-487e-bba0-3ede13ff5411 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:41.750458 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3188ca7-76b6-487e-bba0-3ede13ff5411 None None] PATH_INFO: `/v3` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:41.752106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 769/1514] 10.222.0.48 () {54 vars in 925 bytes} [Mon Jul 25 08:43:41 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:43:41.758513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac1eea-2b61-4526-83aa-6a83255141ce None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:41.758937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac1eea-2b61-4526-83aa-6a83255141ce None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:41.759253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac1eea-2b61-4526-83aa-6a83255141ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:41.823755 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-eeac1eea-2b61-4526-83aa-6a83255141ce None None] MFA Rules not processed for user `62bf4ef4fce74f7b91e453f73430d9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:43:41.824922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-eeac1eea-2b61-4526-83aa-6a83255141ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:43:41.891500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:43:41.896706 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 746/1515] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Jul 25 08:43:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:43:42.446124 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe5454-a4e0-43f2-a179-fed1a0cc8710 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:42.446676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe5454-a4e0-43f2-a179-fed1a0cc8710 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:42.446676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe5454-a4e0-43f2-a179-fed1a0cc8710 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:42.448057 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 770/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 08:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:43:42.453626 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596beb23-df31-4193-a712-2886c5e4e1ed None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:42.454049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596beb23-df31-4193-a712-2886c5e4e1ed None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:42.454198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596beb23-df31-4193-a712-2886c5e4e1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:42.473591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-596beb23-df31-4193-a712-2886c5e4e1ed None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:43:42.474815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-596beb23-df31-4193-a712-2886c5e4e1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:43:42.482791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 747/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 08:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:43:42.634862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe5454-a4e0-43f2-a179-fed1a0cc8710 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:43:42.649745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe5454-a4e0-43f2-a179-fed1a0cc8710 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:43:42.655487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c442c-d5f4-46e7-acb4-c5201a926149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:43:42.656127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c442c-d5f4-46e7-acb4-c5201a926149 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:42.656421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c442c-d5f4-46e7-acb4-c5201a926149 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:42.656751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c442c-d5f4-46e7-acb4-c5201a926149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:42.665039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c442c-d5f4-46e7-acb4-c5201a926149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:43:42.666616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c442c-d5f4-46e7-acb4-c5201a926149 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:43:42.676513 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 771/1518] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:43:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:43:51.047650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a5466-3966-45d4-8585-c5ef81a3aa5b None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:51.048186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a5466-3966-45d4-8585-c5ef81a3aa5b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:51.048186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a5466-3966-45d4-8585-c5ef81a3aa5b None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:51.049593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 748/1519] 10.222.0.52 () {54 vars in 961 bytes} [Mon Jul 25 08:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:43:51.054569 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b61bd-d69e-41b9-8363-54efc72cd213 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:51.055009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b61bd-d69e-41b9-8363-54efc72cd213 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:51.055130 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b61bd-d69e-41b9-8363-54efc72cd213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:51.074752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a07b61bd-d69e-41b9-8363-54efc72cd213 None None] MFA Rules not processed for user `62bf4ef4fce74f7b91e453f73430d9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:43:51.075946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a07b61bd-d69e-41b9-8363-54efc72cd213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:43:51.085139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 772/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Mon Jul 25 08:43:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:43:51.092436 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a5466-3966-45d4-8585-c5ef81a3aa5b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:43:51.105729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a5466-3966-45d4-8585-c5ef81a3aa5b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:43:51.110686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c20a8-8f60-4fe9-8188-498ea98d8844 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:43:51.111314 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c20a8-8f60-4fe9-8188-498ea98d8844 service placement] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:51.111601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c20a8-8f60-4fe9-8188-498ea98d8844 service placement] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:51.111878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c20a8-8f60-4fe9-8188-498ea98d8844 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:51.120074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c20a8-8f60-4fe9-8188-498ea98d8844 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:43:51.121690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c20a8-8f60-4fe9-8188-498ea98d8844 service placement] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:43:51.131784 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 749/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:43:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:43:53.846321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c026c03-c22b-4f7c-85d5-3d623d39edb6 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:53.846906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c026c03-c22b-4f7c-85d5-3d623d39edb6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:53.846906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c026c03-c22b-4f7c-85d5-3d623d39edb6 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:53.848275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 08:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:43:53.853050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592e757-dfdc-4dcb-a624-82227d1c4db3 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:53.853452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592e757-dfdc-4dcb-a624-82227d1c4db3 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:53.853649 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592e757-dfdc-4dcb-a624-82227d1c4db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:53.873463 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-8592e757-dfdc-4dcb-a624-82227d1c4db3 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:43:53.874653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8592e757-dfdc-4dcb-a624-82227d1c4db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:43:53.883160 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 08:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:43:54.036976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c026c03-c22b-4f7c-85d5-3d623d39edb6 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:43:54.052431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c026c03-c22b-4f7c-85d5-3d623d39edb6 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:43:54.058053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8848c-e68f-4f8c-936e-c379adefa8d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:43:54.058697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8848c-e68f-4f8c-936e-c379adefa8d6 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:54.058986 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8848c-e68f-4f8c-936e-c379adefa8d6 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:54.059268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8848c-e68f-4f8c-936e-c379adefa8d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:54.067757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8848c-e68f-4f8c-936e-c379adefa8d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:43:54.069376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8848c-e68f-4f8c-936e-c379adefa8d6 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:43:54.079022 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 774/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:43:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:43:54.897031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bb0b2-c97e-4d75-8f58-04f5fcb974a1 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:54.897581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bb0b2-c97e-4d75-8f58-04f5fcb974a1 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:54.897581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bb0b2-c97e-4d75-8f58-04f5fcb974a1 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:54.898973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 08:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:43:54.903837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b48517-6183-4a10-9f27-c52fa1a18191 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:54.904238 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b48517-6183-4a10-9f27-c52fa1a18191 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:54.904402 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b48517-6183-4a10-9f27-c52fa1a18191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:54.923610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d1b48517-6183-4a10-9f27-c52fa1a18191 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:43:54.924852 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d1b48517-6183-4a10-9f27-c52fa1a18191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:43:54.932765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 08:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:43:55.083501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bb0b2-c97e-4d75-8f58-04f5fcb974a1 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:43:55.097764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bb0b2-c97e-4d75-8f58-04f5fcb974a1 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:43:55.103125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63149b42-f60b-43d0-a121-66118c31fb26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:43:55.103789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63149b42-f60b-43d0-a121-66118c31fb26 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:55.104071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63149b42-f60b-43d0-a121-66118c31fb26 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:55.104346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63149b42-f60b-43d0-a121-66118c31fb26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:55.112779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63149b42-f60b-43d0-a121-66118c31fb26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:43:55.114344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63149b42-f60b-43d0-a121-66118c31fb26 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:43:55.123505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 752/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:43:59.115392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb588db-ac18-4cbf-adf8-ede65e1dbaed None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:59.115975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb588db-ac18-4cbf-adf8-ede65e1dbaed None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:59.116083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb588db-ac18-4cbf-adf8-ede65e1dbaed None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:59.117494 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:43:59.122639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee30bb-ca5d-44e9-9fac-dc4c2c63ee5e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:59.123020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee30bb-ca5d-44e9-9fac-dc4c2c63ee5e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:59.123196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee30bb-ca5d-44e9-9fac-dc4c2c63ee5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:59.142922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-acee30bb-ca5d-44e9-9fac-dc4c2c63ee5e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:43:59.144094 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-acee30bb-ca5d-44e9-9fac-dc4c2c63ee5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:43:59.151828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:43:59.223730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb588db-ac18-4cbf-adf8-ede65e1dbaed None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:43:59.225547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0fb588db-ac18-4cbf-adf8-ede65e1dbaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:43:59.258754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:43:59.309270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb588db-ac18-4cbf-adf8-ede65e1dbaed None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:43:59.312918 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1e12f-1a61-4327-868e-bde8dbfa89d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:43:59.313570 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1e12f-1a61-4327-868e-bde8dbfa89d5 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:43:59.314355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1e12f-1a61-4327-868e-bde8dbfa89d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:43:59.314634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1e12f-1a61-4327-868e-bde8dbfa89d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:43:59.323529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1e12f-1a61-4327-868e-bde8dbfa89d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:43:59.325022 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1e12f-1a61-4327-868e-bde8dbfa89d5 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:43:59.335053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 777/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:43:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:44:01.307416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2b086-44a0-472f-a81b-aba1cab55c25 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:44:01.307812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2b086-44a0-472f-a81b-aba1cab55c25 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:44:01.307979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2b086-44a0-472f-a81b-aba1cab55c25 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:44:01.309430 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:44:01.315311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5601c7-2c33-4f62-adb4-eb7eb29db5dc None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:44:01.315691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5601c7-2c33-4f62-adb4-eb7eb29db5dc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:44:01.315895 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5601c7-2c33-4f62-adb4-eb7eb29db5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:44:01.335230 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4e5601c7-2c33-4f62-adb4-eb7eb29db5dc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:44:01.336413 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4e5601c7-2c33-4f62-adb4-eb7eb29db5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:44:01.344286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:44:01.379844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2b086-44a0-472f-a81b-aba1cab55c25 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:44:01.390832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2b086-44a0-472f-a81b-aba1cab55c25 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:44:01.394621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c1845-7e40-4281-b863-b442a364b547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:44:01.395265 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c1845-7e40-4281-b863-b442a364b547 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:44:01.395548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c1845-7e40-4281-b863-b442a364b547 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:44:01.395824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c1845-7e40-4281-b863-b442a364b547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:44:01.404827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c1845-7e40-4281-b863-b442a364b547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:44:01.406246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c1845-7e40-4281-b863-b442a364b547 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:44:01.415621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 755/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:44:05.113068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661805d8-40ac-48c3-8009-ee3788a5dfea None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:44:05.113732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661805d8-40ac-48c3-8009-ee3788a5dfea None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:44:05.113732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661805d8-40ac-48c3-8009-ee3788a5dfea None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:44:05.115025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:44:05.120119 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682c409-2829-4043-938b-ba773352b886 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:44:05.120527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682c409-2829-4043-938b-ba773352b886 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:44:05.120726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682c409-2829-4043-938b-ba773352b886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:44:05.140612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-4682c409-2829-4043-938b-ba773352b886 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:44:05.142387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4682c409-2829-4043-938b-ba773352b886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:44:05.150334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:44:05.452485 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661805d8-40ac-48c3-8009-ee3788a5dfea None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:44:05.454187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-661805d8-40ac-48c3-8009-ee3788a5dfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:44:05.533476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661805d8-40ac-48c3-8009-ee3788a5dfea None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:44:05.537122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc09810-c867-480d-9fd4-20b2f4768fc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:44:05.537763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc09810-c867-480d-9fd4-20b2f4768fc9 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:44:05.538051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc09810-c867-480d-9fd4-20b2f4768fc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:44:05.538327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc09810-c867-480d-9fd4-20b2f4768fc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:44:05.547376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc09810-c867-480d-9fd4-20b2f4768fc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:44:05.548826 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc09810-c867-480d-9fd4-20b2f4768fc9 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:44:05.558101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 780/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:44:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:44:07.558515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570a700-deeb-423c-b072-713d32c8de89 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:44:07.559039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570a700-deeb-423c-b072-713d32c8de89 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:44:07.559039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570a700-deeb-423c-b072-713d32c8de89 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:44:07.560388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 08:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:44:07.565521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73b11b-cfeb-43da-a23c-f19737aa8fc8 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:44:07.565970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73b11b-cfeb-43da-a23c-f19737aa8fc8 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:44:07.566075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73b11b-cfeb-43da-a23c-f19737aa8fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:44:07.585208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7e73b11b-cfeb-43da-a23c-f19737aa8fc8 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:44:07.586394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7e73b11b-cfeb-43da-a23c-f19737aa8fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:44:07.594708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 08:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:44:07.896630 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570a700-deeb-423c-b072-713d32c8de89 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:44:07.898297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0570a700-deeb-423c-b072-713d32c8de89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:44:07.912148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570a700-deeb-423c-b072-713d32c8de89 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:44:07.915681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3902b43-6924-4992-8ca7-6fcca85313eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:44:07.916318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902b43-6924-4992-8ca7-6fcca85313eb service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:44:07.916633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902b43-6924-4992-8ca7-6fcca85313eb service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:44:07.916916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902b43-6924-4992-8ca7-6fcca85313eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:44:07.925749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3902b43-6924-4992-8ca7-6fcca85313eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:44:07.927158 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3902b43-6924-4992-8ca7-6fcca85313eb service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:44:07.936665 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 758/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:44:17.580200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=63 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:44:17.932965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=37 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:37.294348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d9c57-ad35-4a49-9163-e85c34594de8 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.296715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d9c57-ad35-4a49-9163-e85c34594de8 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.296715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d9c57-ad35-4a49-9163-e85c34594de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.325846 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c99d9c57-ad35-4a49-9163-e85c34594de8 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:37.327444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c99d9c57-ad35-4a49-9163-e85c34594de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:37.343908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 782/1540] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:37.353461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf94b9-599c-426b-82c4-5290861b60cc None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.354020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf94b9-599c-426b-82c4-5290861b60cc None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.354020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf94b9-599c-426b-82c4-5290861b60cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.387090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-14cf94b9-599c-426b-82c4-5290861b60cc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:37.388327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-14cf94b9-599c-426b-82c4-5290861b60cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:37.405535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 759/1541] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:37.411001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d9c57-ad35-4a49-9163-e85c34594de8 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:37.426818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d9c57-ad35-4a49-9163-e85c34594de8 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:37.434150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d87c0e-b60d-4395-8ec0-f3b181904325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:37.435015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d87c0e-b60d-4395-8ec0-f3b181904325 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.435459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d87c0e-b60d-4395-8ec0-f3b181904325 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.435752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d87c0e-b60d-4395-8ec0-f3b181904325 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.436503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d87c0e-b60d-4395-8ec0-f3b181904325 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:45:37.436853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d87c0e-b60d-4395-8ec0-f3b181904325 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:37.439073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d87c0e-b60d-4395-8ec0-f3b181904325 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:37.458226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 783/1542] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:45:37.470458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf94b9-599c-426b-82c4-5290861b60cc None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:37.472503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74688918-3296-44e0-aec9-09fd3b9b3ab2 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.472833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74688918-3296-44e0-aec9-09fd3b9b3ab2 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.473119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74688918-3296-44e0-aec9-09fd3b9b3ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.486294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf94b9-599c-426b-82c4-5290861b60cc None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:37.501343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-74688918-3296-44e0-aec9-09fd3b9b3ab2 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:37.502020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01312af-8279-416f-ac08-bc66a1ad190f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:37.502571 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-74688918-3296-44e0-aec9-09fd3b9b3ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:37.502697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01312af-8279-416f-ac08-bc66a1ad190f admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.502979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01312af-8279-416f-ac08-bc66a1ad190f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.503312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01312af-8279-416f-ac08-bc66a1ad190f admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.504022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01312af-8279-416f-ac08-bc66a1ad190f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:37.506201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01312af-8279-416f-ac08-bc66a1ad190f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:37.515860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 784/1543] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:45:37.524832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6913da3-a8f5-493e-87c4-09686fc6bcb3 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.525123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6913da3-a8f5-493e-87c4-09686fc6bcb3 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.525405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6913da3-a8f5-493e-87c4-09686fc6bcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.556801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-b6913da3-a8f5-493e-87c4-09686fc6bcb3 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:37.556801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b6913da3-a8f5-493e-87c4-09686fc6bcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:37.573895 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 785/1544] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:45:37.578386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6913da3-a8f5-493e-87c4-09686fc6bcb3 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:37.578657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 760/1545] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/projects => generated 393 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:37.583535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01312af-8279-416f-ac08-bc66a1ad190f admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:37.591982 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6913da3-a8f5-493e-87c4-09686fc6bcb3 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:37.699324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01312af-8279-416f-ac08-bc66a1ad190f admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:37.712449 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72253be5-59ab-45d0-8c0e-73b0422bd95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:37.713811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72253be5-59ab-45d0-8c0e-73b0422bd95f admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.714305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72253be5-59ab-45d0-8c0e-73b0422bd95f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.714795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72253be5-59ab-45d0-8c0e-73b0422bd95f admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.716130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72253be5-59ab-45d0-8c0e-73b0422bd95f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:37.718106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: WARNING py.warnings [None req-72253be5-59ab-45d0-8c0e-73b0422bd95f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:45:37.718106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: warnings.warn(msg) Jul 25 08:45:37.718106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: Jul 25 08:45:37.720278 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72253be5-59ab-45d0-8c0e-73b0422bd95f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:37.750377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726331a-0891-477e-8fc5-7fc9a5e70be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:37.751672 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726331a-0891-477e-8fc5-7fc9a5e70be4 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.751965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726331a-0891-477e-8fc5-7fc9a5e70be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.752245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726331a-0891-477e-8fc5-7fc9a5e70be4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.753034 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726331a-0891-477e-8fc5-7fc9a5e70be4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:45:37.753361 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726331a-0891-477e-8fc5-7fc9a5e70be4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:37.754968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726331a-0891-477e-8fc5-7fc9a5e70be4 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:37.761954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 761/1546] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:45:37.763597 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69222c0d-6516-428e-a473-ba218c3b9f60 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.764129 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69222c0d-6516-428e-a473-ba218c3b9f60 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.764790 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69222c0d-6516-428e-a473-ba218c3b9f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.780074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 786/1547] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:45:37.781562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb4d9d-272d-4007-b592-99ca09816e20 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.781889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb4d9d-272d-4007-b592-99ca09816e20 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.782187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb4d9d-272d-4007-b592-99ca09816e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.792099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-69222c0d-6516-428e-a473-ba218c3b9f60 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:37.793889 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-69222c0d-6516-428e-a473-ba218c3b9f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:37.811329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 762/1548] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:37.812101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-aceb4d9d-272d-4007-b592-99ca09816e20 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:37.813211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e89da-5cc7-495e-9a7f-db332da1e2bb None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.813906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-aceb4d9d-272d-4007-b592-99ca09816e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:37.814520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e89da-5cc7-495e-9a7f-db332da1e2bb None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.814971 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e89da-5cc7-495e-9a7f-db332da1e2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.823185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 787/1549] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:37.825792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb4d9d-272d-4007-b592-99ca09816e20 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:37.837016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-4f7e89da-5cc7-495e-9a7f-db332da1e2bb None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:37.837951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb4d9d-272d-4007-b592-99ca09816e20 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:37.838223 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4f7e89da-5cc7-495e-9a7f-db332da1e2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:37.842612 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1af40-073c-494d-a64a-bad9d7efe27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:37.843399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1af40-073c-494d-a64a-bad9d7efe27f admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.843680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1af40-073c-494d-a64a-bad9d7efe27f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.843958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1af40-073c-494d-a64a-bad9d7efe27f admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.844933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1af40-073c-494d-a64a-bad9d7efe27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:37.846543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1af40-073c-494d-a64a-bad9d7efe27f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:37.846661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 763/1550] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:37.851605 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e89da-5cc7-495e-9a7f-db332da1e2bb None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:37.858031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e89da-5cc7-495e-9a7f-db332da1e2bb None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:37.863676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330d574-2df7-4897-8a36-c1ff6497c0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:37.864335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330d574-2df7-4897-8a36-c1ff6497c0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.864635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330d574-2df7-4897-8a36-c1ff6497c0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.864916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330d574-2df7-4897-8a36-c1ff6497c0b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.865607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330d574-2df7-4897-8a36-c1ff6497c0b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:37.866212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 788/1551] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:37.869484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac9f5c-5c53-4382-9f8a-cbd086b79903 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.869633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330d574-2df7-4897-8a36-c1ff6497c0b5 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:37.869722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac9f5c-5c53-4382-9f8a-cbd086b79903 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.869722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac9f5c-5c53-4382-9f8a-cbd086b79903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.891970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-19ac9f5c-5c53-4382-9f8a-cbd086b79903 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:37.893320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-19ac9f5c-5c53-4382-9f8a-cbd086b79903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:37.901981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 789/1552] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:37.905776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 764/1553] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:37.905776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befbcd4-87b7-439b-a1fd-fc9031c26b03 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.905776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befbcd4-87b7-439b-a1fd-fc9031c26b03 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.905776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befbcd4-87b7-439b-a1fd-fc9031c26b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.906187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b8fa6-bfda-4b7b-adb1-ba4217f9b37c None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.906483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b8fa6-bfda-4b7b-adb1-ba4217f9b37c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.906776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b8fa6-bfda-4b7b-adb1-ba4217f9b37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.924412 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5befbcd4-87b7-439b-a1fd-fc9031c26b03 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:37.925718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5befbcd4-87b7-439b-a1fd-fc9031c26b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:37.927240 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c30b8fa6-bfda-4b7b-adb1-ba4217f9b37c None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:37.928421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c30b8fa6-bfda-4b7b-adb1-ba4217f9b37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:37.983442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 765/1554] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:37.984816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98beb3-de36-4e81-ac11-044ee04d4b83 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.985627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98beb3-de36-4e81-ac11-044ee04d4b83 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.985905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98beb3-de36-4e81-ac11-044ee04d4b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:37.987338 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 790/1555] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:37.993931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1070e0-d0c2-4bdf-bff7-c877058afa44 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:37.994688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1070e0-d0c2-4bdf-bff7-c877058afa44 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:37.994974 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1070e0-d0c2-4bdf-bff7-c877058afa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.008089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5a98beb3-de36-4e81-ac11-044ee04d4b83 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:38.009345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5a98beb3-de36-4e81-ac11-044ee04d4b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:38.014996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-3c1070e0-d0c2-4bdf-bff7-c877058afa44 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:38.016159 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3c1070e0-d0c2-4bdf-bff7-c877058afa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:38.017620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 766/1556] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:38.018388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98beb3-de36-4e81-ac11-044ee04d4b83 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.025340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 791/1557] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:38.026618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248d492-7c62-47f5-90b0-1bcde2d64b5c None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.026906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248d492-7c62-47f5-90b0-1bcde2d64b5c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.027176 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248d492-7c62-47f5-90b0-1bcde2d64b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.031452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98beb3-de36-4e81-ac11-044ee04d4b83 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.036479 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114a31e-0700-4f22-bb42-364c8fa60b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.037239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114a31e-0700-4f22-bb42-364c8fa60b62 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.037525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114a31e-0700-4f22-bb42-364c8fa60b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.037811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114a31e-0700-4f22-bb42-364c8fa60b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.038568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114a31e-0700-4f22-bb42-364c8fa60b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.040311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114a31e-0700-4f22-bb42-364c8fa60b62 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.045835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:38.046056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-0248d492-7c62-47f5-90b0-1bcde2d64b5c None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:38.047261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0248d492-7c62-47f5-90b0-1bcde2d64b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:38.052179 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 767/1558] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:38.053342 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114a31e-0700-4f22-bb42-364c8fa60b62 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.059025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 792/1559] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:38.059025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149ba56-20b4-4f2c-add2-92f5bc99a0d6 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.059025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149ba56-20b4-4f2c-add2-92f5bc99a0d6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.059025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149ba56-20b4-4f2c-add2-92f5bc99a0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.065270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114a31e-0700-4f22-bb42-364c8fa60b62 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.070133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba86c9-2804-48ad-be17-c99853723773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.070821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba86c9-2804-48ad-be17-c99853723773 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.071106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba86c9-2804-48ad-be17-c99853723773 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.071391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba86c9-2804-48ad-be17-c99853723773 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.072082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba86c9-2804-48ad-be17-c99853723773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.074086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba86c9-2804-48ad-be17-c99853723773 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.081063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-8149ba56-20b4-4f2c-add2-92f5bc99a0d6 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:38.082253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8149ba56-20b4-4f2c-add2-92f5bc99a0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:38.092437 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 793/1560] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:38.093206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149ba56-20b4-4f2c-add2-92f5bc99a0d6 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.102322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 768/1561] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:38.102322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fa33e-6a91-4afb-ac28-2b049b271262 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.102322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fa33e-6a91-4afb-ac28-2b049b271262 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.102322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fa33e-6a91-4afb-ac28-2b049b271262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.106914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149ba56-20b4-4f2c-add2-92f5bc99a0d6 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.112355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed1c40-2202-4c92-b465-be016f4787cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.113030 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed1c40-2202-4c92-b465-be016f4787cb admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.113545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed1c40-2202-4c92-b465-be016f4787cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.113833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed1c40-2202-4c92-b465-be016f4787cb admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.114697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed1c40-2202-4c92-b465-be016f4787cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:45:38.115006 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed1c40-2202-4c92-b465-be016f4787cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.116617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed1c40-2202-4c92-b465-be016f4787cb admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.121162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-155fa33e-6a91-4afb-ac28-2b049b271262 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:38.122369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-155fa33e-6a91-4afb-ac28-2b049b271262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:38.135797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 794/1562] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:45:38.135797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed1c40-2202-4c92-b465-be016f4787cb admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.136056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 769/1563] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:38.136056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fa33e-6a91-4afb-ac28-2b049b271262 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.138242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:38.141746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed1c40-2202-4c92-b465-be016f4787cb admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.144624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fa33e-6a91-4afb-ac28-2b049b271262 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.146855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89805e-fc5c-4ecf-b313-3fc9948b23b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.147500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89805e-fc5c-4ecf-b313-3fc9948b23b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.147778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89805e-fc5c-4ecf-b313-3fc9948b23b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.148048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89805e-fc5c-4ecf-b313-3fc9948b23b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.148802 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89805e-fc5c-4ecf-b313-3fc9948b23b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:45:38.149104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89805e-fc5c-4ecf-b313-3fc9948b23b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.149741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d4e4f-0167-4a48-b558-7d48c1ef8e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.150395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d4e4f-0167-4a48-b558-7d48c1ef8e8a admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.150680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d4e4f-0167-4a48-b558-7d48c1ef8e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.150781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89805e-fc5c-4ecf-b313-3fc9948b23b7 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.150957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d4e4f-0167-4a48-b558-7d48c1ef8e8a admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.151685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d4e4f-0167-4a48-b558-7d48c1ef8e8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:45:38.151997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d4e4f-0167-4a48-b558-7d48c1ef8e8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.153504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: WARNING py.warnings [None req-940d4e4f-0167-4a48-b558-7d48c1ef8e8a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:45:38.153504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: warnings.warn(msg) Jul 25 08:45:38.153504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: Jul 25 08:45:38.153932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d4e4f-0167-4a48-b558-7d48c1ef8e8a admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.163019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 795/1564] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:45:38.163933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89805e-fc5c-4ecf-b313-3fc9948b23b7 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.167377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 770/1565] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:45:38.169413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f258a4-8eae-4432-81ac-005ee414e74f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.170487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f258a4-8eae-4432-81ac-005ee414e74f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.170982 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f258a4-8eae-4432-81ac-005ee414e74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.174096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89805e-fc5c-4ecf-b313-3fc9948b23b7 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.178940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226cfd4-d132-4ed6-a0fd-2a15dab63b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.179817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226cfd4-d132-4ed6-a0fd-2a15dab63b42 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.180096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226cfd4-d132-4ed6-a0fd-2a15dab63b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.180367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226cfd4-d132-4ed6-a0fd-2a15dab63b42 admin admin] PATH_INFO: `/v3/projects/7e61cbb537384f549eee9ec4c92d9d77/users/238f78244df546e5a6fc1a4cd1b73288/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.192874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-03f258a4-8eae-4432-81ac-005ee414e74f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:38.194087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-03f258a4-8eae-4432-81ac-005ee414e74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:38.195444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b226cfd4-d132-4ed6-a0fd-2a15dab63b42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e61cbb537384f549eee9ec4c92d9d77, user_id=238f78244df546e5a6fc1a4cd1b73288, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.196471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: WARNING py.warnings [None req-b226cfd4-d132-4ed6-a0fd-2a15dab63b42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:45:38.196471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: warnings.warn(msg) Jul 25 08:45:38.196471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: Jul 25 08:45:38.198497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b226cfd4-d132-4ed6-a0fd-2a15dab63b42 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.202074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 771/1566] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:38.202926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f258a4-8eae-4432-81ac-005ee414e74f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.213235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f258a4-8eae-4432-81ac-005ee414e74f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.215260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 796/1567] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:38 2022] PUT /identity/v3/projects/7e61cbb537384f549eee9ec4c92d9d77/users/238f78244df546e5a6fc1a4cd1b73288/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:38.216064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226cfd4-d132-4ed6-a0fd-2a15dab63b42 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.272128 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b41c1e-4395-4cf0-9549-d39d45f6efd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.272832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b41c1e-4395-4cf0-9549-d39d45f6efd9 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.273120 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b41c1e-4395-4cf0-9549-d39d45f6efd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.273399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b41c1e-4395-4cf0-9549-d39d45f6efd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.274127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b41c1e-4395-4cf0-9549-d39d45f6efd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:45:38.274432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b41c1e-4395-4cf0-9549-d39d45f6efd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.276041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b41c1e-4395-4cf0-9549-d39d45f6efd9 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.284358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226cfd4-d132-4ed6-a0fd-2a15dab63b42 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.289512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268cc17-3305-41d8-911b-02dc94132a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.290195 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268cc17-3305-41d8-911b-02dc94132a9f admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.290483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268cc17-3305-41d8-911b-02dc94132a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.290769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268cc17-3305-41d8-911b-02dc94132a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.291506 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268cc17-3305-41d8-911b-02dc94132a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.293162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268cc17-3305-41d8-911b-02dc94132a9f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.295739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 772/1568] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:45:38.296492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b41c1e-4395-4cf0-9549-d39d45f6efd9 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.306950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b41c1e-4395-4cf0-9549-d39d45f6efd9 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.307384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 797/1569] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:38.309480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268cc17-3305-41d8-911b-02dc94132a9f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.314801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c674422-2366-4c4e-b8b8-a10ab2578c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.315536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c674422-2366-4c4e-b8b8-a10ab2578c6e admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.315783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c674422-2366-4c4e-b8b8-a10ab2578c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.316175 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c674422-2366-4c4e-b8b8-a10ab2578c6e admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.316996 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c674422-2366-4c4e-b8b8-a10ab2578c6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:45:38.317396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c674422-2366-4c4e-b8b8-a10ab2578c6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.318998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c674422-2366-4c4e-b8b8-a10ab2578c6e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.321428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268cc17-3305-41d8-911b-02dc94132a9f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.326106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b7410-e2ce-49e9-9bc1-e3351fda1329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.326760 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b7410-e2ce-49e9-9bc1-e3351fda1329 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.327052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b7410-e2ce-49e9-9bc1-e3351fda1329 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.327332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b7410-e2ce-49e9-9bc1-e3351fda1329 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.328031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b7410-e2ce-49e9-9bc1-e3351fda1329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.329171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: WARNING py.warnings [None req-116b7410-e2ce-49e9-9bc1-e3351fda1329 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:45:38.329171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: warnings.warn(msg) Jul 25 08:45:38.329171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: Jul 25 08:45:38.330386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b7410-e2ce-49e9-9bc1-e3351fda1329 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.332115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 773/1570] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:45:38.332900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c674422-2366-4c4e-b8b8-a10ab2578c6e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.344276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c674422-2366-4c4e-b8b8-a10ab2578c6e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.349752 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957aeb5-10ba-4962-8174-c1c4f82ae06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.350534 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957aeb5-10ba-4962-8174-c1c4f82ae06a admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.350849 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957aeb5-10ba-4962-8174-c1c4f82ae06a admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.351138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957aeb5-10ba-4962-8174-c1c4f82ae06a admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.351978 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957aeb5-10ba-4962-8174-c1c4f82ae06a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.353961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957aeb5-10ba-4962-8174-c1c4f82ae06a admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.359923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 798/1571] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:38.364442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b7410-e2ce-49e9-9bc1-e3351fda1329 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.381479 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 774/1572] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:38.382463 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957aeb5-10ba-4962-8174-c1c4f82ae06a admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.431577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b7410-e2ce-49e9-9bc1-e3351fda1329 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.443938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957aeb5-10ba-4962-8174-c1c4f82ae06a admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.448635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411d254-bd94-4937-b79f-60a778b2fcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.449302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411d254-bd94-4937-b79f-60a778b2fcab admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.449588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411d254-bd94-4937-b79f-60a778b2fcab admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.449868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411d254-bd94-4937-b79f-60a778b2fcab admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.450628 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411d254-bd94-4937-b79f-60a778b2fcab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:45:38.450932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411d254-bd94-4937-b79f-60a778b2fcab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.452533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411d254-bd94-4937-b79f-60a778b2fcab admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.463850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 775/1573] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:45:38.470388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8c689-dcdc-464c-b61c-37c293e30afa None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.470794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8c689-dcdc-464c-b61c-37c293e30afa None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.471078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8c689-dcdc-464c-b61c-37c293e30afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.478672 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:38.491974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-04e8c689-dcdc-464c-b61c-37c293e30afa None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:38.493200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-04e8c689-dcdc-464c-b61c-37c293e30afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:38.495511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb9e7a-ce68-46fb-ae78-d92d7494b599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.496189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb9e7a-ce68-46fb-ae78-d92d7494b599 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.496479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb9e7a-ce68-46fb-ae78-d92d7494b599 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.496947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb9e7a-ce68-46fb-ae78-d92d7494b599 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.497658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb9e7a-ce68-46fb-ae78-d92d7494b599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.499580 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb9e7a-ce68-46fb-ae78-d92d7494b599 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.506687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 776/1574] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:38.508285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8c689-dcdc-464c-b61c-37c293e30afa None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.513388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:38.518995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8c689-dcdc-464c-b61c-37c293e30afa None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.525737 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea8637-5a98-4954-9cf1-2028aee33101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.526408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea8637-5a98-4954-9cf1-2028aee33101 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.526697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea8637-5a98-4954-9cf1-2028aee33101 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.526976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea8637-5a98-4954-9cf1-2028aee33101 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.527675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea8637-5a98-4954-9cf1-2028aee33101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.532389 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 799/1575] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/projects => generated 371 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:38.532544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea8637-5a98-4954-9cf1-2028aee33101 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.532674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb9e7a-ce68-46fb-ae78-d92d7494b599 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.558529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 777/1576] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/projects => generated 353 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:38.562631 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea8637-5a98-4954-9cf1-2028aee33101 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.595353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb9e7a-ce68-46fb-ae78-d92d7494b599 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.624141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea8637-5a98-4954-9cf1-2028aee33101 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.629272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7520765-dc7a-4fda-a30a-5bfbf012dead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.629922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7520765-dc7a-4fda-a30a-5bfbf012dead admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.630207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7520765-dc7a-4fda-a30a-5bfbf012dead admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.630477 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7520765-dc7a-4fda-a30a-5bfbf012dead admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.631163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7520765-dc7a-4fda-a30a-5bfbf012dead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.633080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7520765-dc7a-4fda-a30a-5bfbf012dead admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.652379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9ffcd-5f02-40bd-833d-739924d784cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.653094 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9ffcd-5f02-40bd-833d-739924d784cd admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.653391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9ffcd-5f02-40bd-833d-739924d784cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.653674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9ffcd-5f02-40bd-833d-739924d784cd admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.654406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9ffcd-5f02-40bd-833d-739924d784cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.656008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9ffcd-5f02-40bd-833d-739924d784cd admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.663480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 778/1577] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:38.663480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7520765-dc7a-4fda-a30a-5bfbf012dead admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.671981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 800/1578] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:38.671981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9ffcd-5f02-40bd-833d-739924d784cd admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.679532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9ffcd-5f02-40bd-833d-739924d784cd admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.722098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7520765-dc7a-4fda-a30a-5bfbf012dead admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.727199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc403856-06f9-43db-9dfe-0ed74f70430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.727912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc403856-06f9-43db-9dfe-0ed74f70430f admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.728200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc403856-06f9-43db-9dfe-0ed74f70430f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.728512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc403856-06f9-43db-9dfe-0ed74f70430f admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.729272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc403856-06f9-43db-9dfe-0ed74f70430f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.731233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc403856-06f9-43db-9dfe-0ed74f70430f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.736157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c31753-35e4-4caf-8f1c-28a061e444d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.736869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c31753-35e4-4caf-8f1c-28a061e444d2 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.737158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c31753-35e4-4caf-8f1c-28a061e444d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.737439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c31753-35e4-4caf-8f1c-28a061e444d2 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.738161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c31753-35e4-4caf-8f1c-28a061e444d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.740075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c31753-35e4-4caf-8f1c-28a061e444d2 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.752917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 779/1579] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:38.753699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc403856-06f9-43db-9dfe-0ed74f70430f admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.761943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 801/1580] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:38.762773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c31753-35e4-4caf-8f1c-28a061e444d2 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.765448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc403856-06f9-43db-9dfe-0ed74f70430f admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.770913 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3010c-be36-4618-bf9f-79c37de2b21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.771568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3010c-be36-4618-bf9f-79c37de2b21e admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.771856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3010c-be36-4618-bf9f-79c37de2b21e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.772138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3010c-be36-4618-bf9f-79c37de2b21e admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.772865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3010c-be36-4618-bf9f-79c37de2b21e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.773395 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:38.774815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3010c-be36-4618-bf9f-79c37de2b21e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.804614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 780/1581] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:38.804614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3010c-be36-4618-bf9f-79c37de2b21e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.825503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c31753-35e4-4caf-8f1c-28a061e444d2 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.839253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:38.868601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3010c-be36-4618-bf9f-79c37de2b21e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.873280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c543c-443a-4fd1-9715-2084d83beff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.873945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c543c-443a-4fd1-9715-2084d83beff7 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.874227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c543c-443a-4fd1-9715-2084d83beff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.874500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c543c-443a-4fd1-9715-2084d83beff7 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.875194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c543c-443a-4fd1-9715-2084d83beff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.875425 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2c0d8-fb16-4ead-a16f-fa84135a67cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.876118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2c0d8-fb16-4ead-a16f-fa84135a67cb service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.876410 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2c0d8-fb16-4ead-a16f-fa84135a67cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.876723 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2c0d8-fb16-4ead-a16f-fa84135a67cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.877168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c543c-443a-4fd1-9715-2084d83beff7 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.886054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2c0d8-fb16-4ead-a16f-fa84135a67cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.887752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2c0d8-fb16-4ead-a16f-fa84135a67cb service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.898378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 802/1582] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:38.899345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 781/1583] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:38.899555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2c0d8-fb16-4ead-a16f-fa84135a67cb service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.900420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c543c-443a-4fd1-9715-2084d83beff7 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.910925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2c0d8-fb16-4ead-a16f-fa84135a67cb service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.911083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c543c-443a-4fd1-9715-2084d83beff7 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.916062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7af82d-e08a-4dba-8a40-d54fbcc7ecfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.916483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8061d6-0e97-4dc7-a319-5e98e82a4a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.916769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7af82d-e08a-4dba-8a40-d54fbcc7ecfe admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.917057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7af82d-e08a-4dba-8a40-d54fbcc7ecfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.917172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8061d6-0e97-4dc7-a319-5e98e82a4a24 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.917354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7af82d-e08a-4dba-8a40-d54fbcc7ecfe admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.917494 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8061d6-0e97-4dc7-a319-5e98e82a4a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.917730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8061d6-0e97-4dc7-a319-5e98e82a4a24 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.918188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7af82d-e08a-4dba-8a40-d54fbcc7ecfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.918434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8061d6-0e97-4dc7-a319-5e98e82a4a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.920160 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7af82d-e08a-4dba-8a40-d54fbcc7ecfe admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.920388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8061d6-0e97-4dc7-a319-5e98e82a4a24 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.947950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 803/1584] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/users => generated 277 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:38.947950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8061d6-0e97-4dc7-a319-5e98e82a4a24 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.948199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 782/1585] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:38 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:38.948199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7af82d-e08a-4dba-8a40-d54fbcc7ecfe admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.958341 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7af82d-e08a-4dba-8a40-d54fbcc7ecfe admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.958676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8061d6-0e97-4dc7-a319-5e98e82a4a24 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.963132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311580d1-e19f-4286-87b3-5a49d644915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.963803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311580d1-e19f-4286-87b3-5a49d644915e admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.964459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906468b-4a26-4441-ad4f-53a315232b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.964695 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311580d1-e19f-4286-87b3-5a49d644915e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.964695 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311580d1-e19f-4286-87b3-5a49d644915e admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.964960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906468b-4a26-4441-ad4f-53a315232b85 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.965163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311580d1-e19f-4286-87b3-5a49d644915e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.965285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906468b-4a26-4441-ad4f-53a315232b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.965607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906468b-4a26-4441-ad4f-53a315232b85 admin admin] PATH_INFO: `/v3/projects/9313fe06660f4049be47f901c3fc6f76/users/718ca333456f4de5b98f191ab40cca0b/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.966738 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311580d1-e19f-4286-87b3-5a49d644915e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.978041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 783/1586] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:38.978947 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311580d1-e19f-4286-87b3-5a49d644915e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:38.985972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906468b-4a26-4441-ad4f-53a315232b85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9313fe06660f4049be47f901c3fc6f76, user_id=718ca333456f4de5b98f191ab40cca0b, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.989117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906468b-4a26-4441-ad4f-53a315232b85 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:38.989425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311580d1-e19f-4286-87b3-5a49d644915e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:38.994417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b240daf-1f6b-403a-9df7-260208096fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:38.995082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b240daf-1f6b-403a-9df7-260208096fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:38.995414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b240daf-1f6b-403a-9df7-260208096fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:38.995749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b240daf-1f6b-403a-9df7-260208096fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:38.996893 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b240daf-1f6b-403a-9df7-260208096fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:38.998538 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b240daf-1f6b-403a-9df7-260208096fd8 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.010209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 804/1587] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:38 2022] PUT /identity/v3/projects/9313fe06660f4049be47f901c3fc6f76/users/718ca333456f4de5b98f191ab40cca0b/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:39.010209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906468b-4a26-4441-ad4f-53a315232b85 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.013280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 784/1588] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:39.017823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b240daf-1f6b-403a-9df7-260208096fd8 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.028453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b240daf-1f6b-403a-9df7-260208096fd8 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.083395 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906468b-4a26-4441-ad4f-53a315232b85 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.092215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00b49f-9b7c-4e42-97bc-93f7ae0e61da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.093360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00b49f-9b7c-4e42-97bc-93f7ae0e61da admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.093656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00b49f-9b7c-4e42-97bc-93f7ae0e61da admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.093886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00b49f-9b7c-4e42-97bc-93f7ae0e61da admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.095038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00b49f-9b7c-4e42-97bc-93f7ae0e61da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.097437 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00b49f-9b7c-4e42-97bc-93f7ae0e61da admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.100727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45470c92-dc49-4620-9733-279e9391d76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.101794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45470c92-dc49-4620-9733-279e9391d76d admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.102123 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45470c92-dc49-4620-9733-279e9391d76d admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.102482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45470c92-dc49-4620-9733-279e9391d76d admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.103567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45470c92-dc49-4620-9733-279e9391d76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.105914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45470c92-dc49-4620-9733-279e9391d76d admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.122403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 785/1589] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:39.122403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45470c92-dc49-4620-9733-279e9391d76d admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.122645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 805/1590] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:39 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:39.123198 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00b49f-9b7c-4e42-97bc-93f7ae0e61da admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.136890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00b49f-9b7c-4e42-97bc-93f7ae0e61da admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.137816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45470c92-dc49-4620-9733-279e9391d76d admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.142334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1b275-4b81-4c69-84b7-7052643a0c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.143001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1b275-4b81-4c69-84b7-7052643a0c34 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.143335 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1b275-4b81-4c69-84b7-7052643a0c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.143469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebef4c-7309-4150-8d71-321f3b61ca53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.143681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1b275-4b81-4c69-84b7-7052643a0c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.144153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebef4c-7309-4150-8d71-321f3b61ca53 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.144345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1b275-4b81-4c69-84b7-7052643a0c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.144644 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebef4c-7309-4150-8d71-321f3b61ca53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.144990 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebef4c-7309-4150-8d71-321f3b61ca53 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.146216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebef4c-7309-4150-8d71-321f3b61ca53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.146350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1b275-4b81-4c69-84b7-7052643a0c34 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.147491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebef4c-7309-4150-8d71-321f3b61ca53 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.160615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 806/1591] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:39.160615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1b275-4b81-4c69-84b7-7052643a0c34 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.160962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 786/1592] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:39.160962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dce1b9-0162-4831-ade8-b90c308a7d7f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.160962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dce1b9-0162-4831-ade8-b90c308a7d7f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.161185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dce1b9-0162-4831-ade8-b90c308a7d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.166626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:39.172266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1b275-4b81-4c69-84b7-7052643a0c34 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.178198 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370009a-8015-4b43-8522-81596a2a52c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.178941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370009a-8015-4b43-8522-81596a2a52c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.179565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370009a-8015-4b43-8522-81596a2a52c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.180100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370009a-8015-4b43-8522-81596a2a52c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.180930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370009a-8015-4b43-8522-81596a2a52c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.182628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370009a-8015-4b43-8522-81596a2a52c8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.186690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-92dce1b9-0162-4831-ade8-b90c308a7d7f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:39.187924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-92dce1b9-0162-4831-ade8-b90c308a7d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:39.200149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 787/1593] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:39.200149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dce1b9-0162-4831-ade8-b90c308a7d7f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.200436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 807/1594] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:39.208267 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370009a-8015-4b43-8522-81596a2a52c8 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.209801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dce1b9-0162-4831-ade8-b90c308a7d7f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.214859 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3859825-1863-43e6-b8ae-d436a3c60f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.216194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3859825-1863-43e6-b8ae-d436a3c60f07 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.216498 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3859825-1863-43e6-b8ae-d436a3c60f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.216800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3859825-1863-43e6-b8ae-d436a3c60f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.217984 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3859825-1863-43e6-b8ae-d436a3c60f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.219188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3859825-1863-43e6-b8ae-d436a3c60f07 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.219663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370009a-8015-4b43-8522-81596a2a52c8 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.228871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af43ddd-f479-42f7-a7d2-adf276c4da2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.230020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af43ddd-f479-42f7-a7d2-adf276c4da2e admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.230157 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 788/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:39.230691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af43ddd-f479-42f7-a7d2-adf276c4da2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.231464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af43ddd-f479-42f7-a7d2-adf276c4da2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.233103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af43ddd-f479-42f7-a7d2-adf276c4da2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.236722 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3859825-1863-43e6-b8ae-d436a3c60f07 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.237197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af43ddd-f479-42f7-a7d2-adf276c4da2e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.250140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3859825-1863-43e6-b8ae-d436a3c60f07 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.256559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 808/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:39.256559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af43ddd-f479-42f7-a7d2-adf276c4da2e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.257626 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c2298-a77f-4b93-83ec-fcb45e0f2082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.258229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c2298-a77f-4b93-83ec-fcb45e0f2082 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.258516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c2298-a77f-4b93-83ec-fcb45e0f2082 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.258791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c2298-a77f-4b93-83ec-fcb45e0f2082 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.259519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c2298-a77f-4b93-83ec-fcb45e0f2082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.261178 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c2298-a77f-4b93-83ec-fcb45e0f2082 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.271086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af43ddd-f479-42f7-a7d2-adf276c4da2e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.281273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f11bc-cf27-418a-8a19-6c5475f493ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.282466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f11bc-cf27-418a-8a19-6c5475f493ce admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.284772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 789/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:39.286279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c2298-a77f-4b93-83ec-fcb45e0f2082 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.291901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f11bc-cf27-418a-8a19-6c5475f493ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.291901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f11bc-cf27-418a-8a19-6c5475f493ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.291901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f11bc-cf27-418a-8a19-6c5475f493ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.291901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f11bc-cf27-418a-8a19-6c5475f493ce admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.299405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c2298-a77f-4b93-83ec-fcb45e0f2082 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.308749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51729ca3-1a3d-4ce4-be53-a178327795eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.309441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51729ca3-1a3d-4ce4-be53-a178327795eb admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.309731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51729ca3-1a3d-4ce4-be53-a178327795eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.310015 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51729ca3-1a3d-4ce4-be53-a178327795eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.310758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51729ca3-1a3d-4ce4-be53-a178327795eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.313227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51729ca3-1a3d-4ce4-be53-a178327795eb admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.314137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 809/1598] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:39.315540 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f11bc-cf27-418a-8a19-6c5475f493ce admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.329489 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f11bc-cf27-418a-8a19-6c5475f493ce admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.336784 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 790/1599] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:39.336951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb7d14-2944-4445-b9a1-4d1f0c7358da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.337098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51729ca3-1a3d-4ce4-be53-a178327795eb admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.337194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb7d14-2944-4445-b9a1-4d1f0c7358da admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.337194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb7d14-2944-4445-b9a1-4d1f0c7358da admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.337194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb7d14-2944-4445-b9a1-4d1f0c7358da admin admin] PATH_INFO: `/v3/projects/662a6c1838d64a028ee6fd1de8d58c2a/users/a18555c9510c453280d50712f2e22bd0/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.339159 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-51729ca3-1a3d-4ce4-be53-a178327795eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:39.354050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb7d14-2944-4445-b9a1-4d1f0c7358da admin admin] RBAC: Authorizing `identity:create_grant(project_id=662a6c1838d64a028ee6fd1de8d58c2a, user_id=a18555c9510c453280d50712f2e22bd0, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.357983 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb7d14-2944-4445-b9a1-4d1f0c7358da admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.373251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 810/1600] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:39 2022] PUT /identity/v3/projects/662a6c1838d64a028ee6fd1de8d58c2a/users/a18555c9510c453280d50712f2e22bd0/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:39.373889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb7d14-2944-4445-b9a1-4d1f0c7358da admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.375432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:39.409382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51729ca3-1a3d-4ce4-be53-a178327795eb admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.447435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb7d14-2944-4445-b9a1-4d1f0c7358da admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.452997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1676d0-62a1-4c5e-9143-8ce89f82d4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.454424 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1676d0-62a1-4c5e-9143-8ce89f82d4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.454809 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1676d0-62a1-4c5e-9143-8ce89f82d4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.455106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1676d0-62a1-4c5e-9143-8ce89f82d4c4 admin admin] PATH_INFO: `/v3/projects/b04bad9c05ec40a4a5975e9e03a1f4d2/users/95487356edb841b387df2a31b519a740/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.472103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c209e-6fa7-4c88-9012-abf599b6878d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.472357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1676d0-62a1-4c5e-9143-8ce89f82d4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04bad9c05ec40a4a5975e9e03a1f4d2, user_id=95487356edb841b387df2a31b519a740, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.472914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c209e-6fa7-4c88-9012-abf599b6878d service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.473258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c209e-6fa7-4c88-9012-abf599b6878d service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.473859 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c209e-6fa7-4c88-9012-abf599b6878d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.475086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1676d0-62a1-4c5e-9143-8ce89f82d4c4 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.486649 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c209e-6fa7-4c88-9012-abf599b6878d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.488171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c209e-6fa7-4c88-9012-abf599b6878d service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.490768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 811/1601] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:39 2022] PUT /identity/v3/projects/b04bad9c05ec40a4a5975e9e03a1f4d2/users/95487356edb841b387df2a31b519a740/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:39.498813 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 791/1602] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:39.499639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c209e-6fa7-4c88-9012-abf599b6878d service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.502900 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1676d0-62a1-4c5e-9143-8ce89f82d4c4 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.509290 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c209e-6fa7-4c88-9012-abf599b6878d service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.573594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1676d0-62a1-4c5e-9143-8ce89f82d4c4 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.577038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0cd7c6-eeb9-4a58-9d07-b382a15ec408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.577720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cd7c6-eeb9-4a58-9d07-b382a15ec408 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.578040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cd7c6-eeb9-4a58-9d07-b382a15ec408 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.578273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cd7c6-eeb9-4a58-9d07-b382a15ec408 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.579231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f6731-0fbe-4f03-91fe-e706f41bb747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.579533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0cd7c6-eeb9-4a58-9d07-b382a15ec408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.580254 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f6731-0fbe-4f03-91fe-e706f41bb747 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.580642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f6731-0fbe-4f03-91fe-e706f41bb747 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.581001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f6731-0fbe-4f03-91fe-e706f41bb747 admin admin] PATH_INFO: `/v3/projects/13ff188603624645a8a7032bb93b21f4/users/fc40b02de0bf43e08070e79440d55480/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.581267 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0cd7c6-eeb9-4a58-9d07-b382a15ec408 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.597364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 792/1603] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:39.597364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0cd7c6-eeb9-4a58-9d07-b382a15ec408 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.601233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f6731-0fbe-4f03-91fe-e706f41bb747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ff188603624645a8a7032bb93b21f4, user_id=fc40b02de0bf43e08070e79440d55480, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.604679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f6731-0fbe-4f03-91fe-e706f41bb747 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.605986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0cd7c6-eeb9-4a58-9d07-b382a15ec408 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.611644 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5668d06-780e-4fe9-bc88-e8b99770e018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.612602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5668d06-780e-4fe9-bc88-e8b99770e018 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.612870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5668d06-780e-4fe9-bc88-e8b99770e018 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.613156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5668d06-780e-4fe9-bc88-e8b99770e018 admin admin] PATH_INFO: `/v3/projects/6924bf00364e4ce9aef4da1854307ac2/users/6dbc3275520844edb074078eb8f54c4b/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.623626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 812/1604] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:39 2022] PUT /identity/v3/projects/13ff188603624645a8a7032bb93b21f4/users/fc40b02de0bf43e08070e79440d55480/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:39.624892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f6731-0fbe-4f03-91fe-e706f41bb747 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.635002 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5668d06-780e-4fe9-bc88-e8b99770e018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6924bf00364e4ce9aef4da1854307ac2, user_id=6dbc3275520844edb074078eb8f54c4b, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.638232 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5668d06-780e-4fe9-bc88-e8b99770e018 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.639201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:39.663109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 793/1605] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:39 2022] PUT /identity/v3/projects/6924bf00364e4ce9aef4da1854307ac2/users/6dbc3275520844edb074078eb8f54c4b/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:39.663109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b542c-e0cd-4fd9-9d99-69f637b7374e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.663109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b542c-e0cd-4fd9-9d99-69f637b7374e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.663109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b542c-e0cd-4fd9-9d99-69f637b7374e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.686543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3f3b542c-e0cd-4fd9-9d99-69f637b7374e None None] MFA Rules not processed for user `a18555c9510c453280d50712f2e22bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:39.687827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3f3b542c-e0cd-4fd9-9d99-69f637b7374e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:39.704269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f6731-0fbe-4f03-91fe-e706f41bb747 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.747088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:39.774005 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 794/1606] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:39.776185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5355e52-284a-4cad-9400-909572c21963 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.776812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5355e52-284a-4cad-9400-909572c21963 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.778435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce03791-2eee-40a3-b8a2-294bdfac133f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:39.779441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce03791-2eee-40a3-b8a2-294bdfac133f admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.779778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce03791-2eee-40a3-b8a2-294bdfac133f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.780083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce03791-2eee-40a3-b8a2-294bdfac133f admin admin] PATH_INFO: `/v3/projects/aa18cc0817c848a782e4176af753e3e8/users/bc39f30010e94eb5862285347575f673/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.780202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5355e52-284a-4cad-9400-909572c21963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.801774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce03791-2eee-40a3-b8a2-294bdfac133f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa18cc0817c848a782e4176af753e3e8, user_id=bc39f30010e94eb5862285347575f673, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:39.805467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce03791-2eee-40a3-b8a2-294bdfac133f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:39.823138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-e5355e52-284a-4cad-9400-909572c21963 None None] MFA Rules not processed for user `95487356edb841b387df2a31b519a740`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:39.824597 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e5355e52-284a-4cad-9400-909572c21963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:39.848373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 813/1607] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:39 2022] PUT /identity/v3/projects/aa18cc0817c848a782e4176af753e3e8/users/bc39f30010e94eb5862285347575f673/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:39.849416 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce03791-2eee-40a3-b8a2-294bdfac133f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:39.937779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 795/1608] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:39 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:39.940380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83a779-38df-4dbe-a3e8-57e0078b9882 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:39.941641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83a779-38df-4dbe-a3e8-57e0078b9882 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:39.943662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83a779-38df-4dbe-a3e8-57e0078b9882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:39.980563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce03791-2eee-40a3-b8a2-294bdfac133f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:39.999602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38612001-64c5-43a0-b213-c275f4bf4406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:40.001290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38612001-64c5-43a0-b213-c275f4bf4406 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:40.001700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38612001-64c5-43a0-b213-c275f4bf4406 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:40.002178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38612001-64c5-43a0-b213-c275f4bf4406 admin admin] PATH_INFO: `/v3/projects/86bea90252bc44b580c40b24b0bd4ed3/users/8e682232b7374450a7265e092602a5a6/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:40.014074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-cd83a779-38df-4dbe-a3e8-57e0078b9882 None None] MFA Rules not processed for user `fc40b02de0bf43e08070e79440d55480`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:40.015441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-cd83a779-38df-4dbe-a3e8-57e0078b9882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:40.026949 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38612001-64c5-43a0-b213-c275f4bf4406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bea90252bc44b580c40b24b0bd4ed3, user_id=8e682232b7374450a7265e092602a5a6, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:40.031780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38612001-64c5-43a0-b213-c275f4bf4406 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:40.069910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 814/1609] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:39 2022] PUT /identity/v3/projects/86bea90252bc44b580c40b24b0bd4ed3/users/8e682232b7374450a7265e092602a5a6/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:40.071762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbc22c-1463-4b2d-ac73-f05ed29d7644 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:40.072205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbc22c-1463-4b2d-ac73-f05ed29d7644 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:40.072602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbc22c-1463-4b2d-ac73-f05ed29d7644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:40.119145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-cfcbc22c-1463-4b2d-ac73-f05ed29d7644 None None] MFA Rules not processed for user `6dbc3275520844edb074078eb8f54c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:40.120859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-cfcbc22c-1463-4b2d-ac73-f05ed29d7644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:40.139223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:40.173600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 796/1610] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:40.173600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83a779-38df-4dbe-a3e8-57e0078b9882 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:40.196199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83a779-38df-4dbe-a3e8-57e0078b9882 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:40.243327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 815/1611] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:40 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:40.243327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da19c77-189c-408c-ba69-5c891a731fd6 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:40.243327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da19c77-189c-408c-ba69-5c891a731fd6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:40.243327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da19c77-189c-408c-ba69-5c891a731fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:40.266066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-9da19c77-189c-408c-ba69-5c891a731fd6 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:40.268078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9da19c77-189c-408c-ba69-5c891a731fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:40.340254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463a519-e8b6-44c3-9d9b-a940ee47ca7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:40.341714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463a519-e8b6-44c3-9d9b-a940ee47ca7a admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:40.342386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463a519-e8b6-44c3-9d9b-a940ee47ca7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:40.342903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463a519-e8b6-44c3-9d9b-a940ee47ca7a admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:40.343982 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463a519-e8b6-44c3-9d9b-a940ee47ca7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:40.347282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463a519-e8b6-44c3-9d9b-a940ee47ca7a admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:40.349905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 816/1612] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:40.352171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da19c77-189c-408c-ba69-5c891a731fd6 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:40.370828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:40.394113 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 797/1613] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:40 2022] POST /identity/v3/projects => generated 403 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:40.398855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9924255-9e04-4a90-b3e8-d2f45748d6d7 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:40.399153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9924255-9e04-4a90-b3e8-d2f45748d6d7 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:40.401084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9924255-9e04-4a90-b3e8-d2f45748d6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:40.430148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-e9924255-9e04-4a90-b3e8-d2f45748d6d7 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:40.431344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e9924255-9e04-4a90-b3e8-d2f45748d6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:40.461975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da19c77-189c-408c-ba69-5c891a731fd6 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:40.513365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 798/1614] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:40.514629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9924255-9e04-4a90-b3e8-d2f45748d6d7 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:40.530537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9924255-9e04-4a90-b3e8-d2f45748d6d7 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:40.544759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3444994-7b5d-4dc4-a990-5dd6d3975c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:40.547024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3444994-7b5d-4dc4-a990-5dd6d3975c7e admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:40.548504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3444994-7b5d-4dc4-a990-5dd6d3975c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:40.548925 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3444994-7b5d-4dc4-a990-5dd6d3975c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:40.551113 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3444994-7b5d-4dc4-a990-5dd6d3975c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:40.555215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3444994-7b5d-4dc4-a990-5dd6d3975c7e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:40.574501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53239298-7531-4adf-aca6-2b8cc299f45a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:40.575341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53239298-7531-4adf-aca6-2b8cc299f45a service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:40.575733 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53239298-7531-4adf-aca6-2b8cc299f45a service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:40.576148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53239298-7531-4adf-aca6-2b8cc299f45a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:40.616111 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:40.627633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 799/1615] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:40 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:40.629343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3444994-7b5d-4dc4-a990-5dd6d3975c7e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:40.674473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53239298-7531-4adf-aca6-2b8cc299f45a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:40.676936 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53239298-7531-4adf-aca6-2b8cc299f45a service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:40.693987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 817/1616] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:40 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:40.697442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53239298-7531-4adf-aca6-2b8cc299f45a service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:40.770092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3444994-7b5d-4dc4-a990-5dd6d3975c7e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:40.776836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45c00e-b068-4d6f-bd16-1aec2a989aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:40.777907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45c00e-b068-4d6f-bd16-1aec2a989aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:40.778354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45c00e-b068-4d6f-bd16-1aec2a989aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:40.779459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45c00e-b068-4d6f-bd16-1aec2a989aa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:40.779880 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45c00e-b068-4d6f-bd16-1aec2a989aa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:40.782054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45c00e-b068-4d6f-bd16-1aec2a989aa6 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:40.820777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53239298-7531-4adf-aca6-2b8cc299f45a service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:40.830751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 800/1617] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:40 2022] POST /identity/v3/projects => generated 355 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:40.832453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45c00e-b068-4d6f-bd16-1aec2a989aa6 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:40.941531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af071ed3-955c-475b-89c6-2af9ddf5c47f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:40.942210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af071ed3-955c-475b-89c6-2af9ddf5c47f service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:40.942562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af071ed3-955c-475b-89c6-2af9ddf5c47f service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:40.942765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af071ed3-955c-475b-89c6-2af9ddf5c47f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:40.955320 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:40.959795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45c00e-b068-4d6f-bd16-1aec2a989aa6 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:40.965328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af071ed3-955c-475b-89c6-2af9ddf5c47f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:40.967062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af071ed3-955c-475b-89c6-2af9ddf5c47f service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:40.967729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba50d68-b749-4b0b-a951-3b44eb13f23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:40.969294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba50d68-b749-4b0b-a951-3b44eb13f23d admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:40.969589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba50d68-b749-4b0b-a951-3b44eb13f23d admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:40.970306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba50d68-b749-4b0b-a951-3b44eb13f23d admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:40.971008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba50d68-b749-4b0b-a951-3b44eb13f23d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:40.973633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba50d68-b749-4b0b-a951-3b44eb13f23d admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:40.991043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 818/1618] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:40.992049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af071ed3-955c-475b-89c6-2af9ddf5c47f service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.032358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af071ed3-955c-475b-89c6-2af9ddf5c47f service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:41.041219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 801/1619] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:40 2022] POST /identity/v3/users => generated 308 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:41.041219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba50d68-b749-4b0b-a951-3b44eb13f23d admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.065132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba50d68-b749-4b0b-a951-3b44eb13f23d admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:41.083779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb4622-1b38-49c9-9877-0f65ced721d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:41.084490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb4622-1b38-49c9-9877-0f65ced721d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.085056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb4622-1b38-49c9-9877-0f65ced721d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.085343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb4622-1b38-49c9-9877-0f65ced721d6 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.086057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb4622-1b38-49c9-9877-0f65ced721d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:41.090634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb4622-1b38-49c9-9877-0f65ced721d6 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:41.123724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b2b0f-022a-41da-a090-23eabe75282a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:41.124779 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b2b0f-022a-41da-a090-23eabe75282a service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.126164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b2b0f-022a-41da-a090-23eabe75282a service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.126912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b2b0f-022a-41da-a090-23eabe75282a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.136843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 802/1620] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:41 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:41.138648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb4622-1b38-49c9-9877-0f65ced721d6 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.145275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b2b0f-022a-41da-a090-23eabe75282a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:41.147399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b2b0f-022a-41da-a090-23eabe75282a service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:41.160731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb4622-1b38-49c9-9877-0f65ced721d6 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:41.162286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 819/1621] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:41.163484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b2b0f-022a-41da-a090-23eabe75282a service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.179347 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1045e-fd3d-4dda-b944-ab99d822358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:41.179863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b2b0f-022a-41da-a090-23eabe75282a service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:41.180085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1045e-fd3d-4dda-b944-ab99d822358d admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.180352 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1045e-fd3d-4dda-b944-ab99d822358d admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.180658 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1045e-fd3d-4dda-b944-ab99d822358d admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.181375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1045e-fd3d-4dda-b944-ab99d822358d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:41.183303 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1045e-fd3d-4dda-b944-ab99d822358d admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:41.188739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b88da6-c83e-4e4b-9dd0-7ea8ac619e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:41.189917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b88da6-c83e-4e4b-9dd0-7ea8ac619e0c admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.190432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b88da6-c83e-4e4b-9dd0-7ea8ac619e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.190950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b88da6-c83e-4e4b-9dd0-7ea8ac619e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.191959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b88da6-c83e-4e4b-9dd0-7ea8ac619e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:41.194118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b88da6-c83e-4e4b-9dd0-7ea8ac619e0c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:41.213543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 820/1622] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:41.215199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b88da6-c83e-4e4b-9dd0-7ea8ac619e0c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.227159 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 803/1623] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:41 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:41.227159 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1045e-fd3d-4dda-b944-ab99d822358d admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.232167 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b88da6-c83e-4e4b-9dd0-7ea8ac619e0c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:41.243744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444246a0-a5db-48ac-9836-677250c7ecdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:41.245142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444246a0-a5db-48ac-9836-677250c7ecdc admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.245722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444246a0-a5db-48ac-9836-677250c7ecdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.246400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1045e-fd3d-4dda-b944-ab99d822358d admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:41.247232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444246a0-a5db-48ac-9836-677250c7ecdc admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.248654 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444246a0-a5db-48ac-9836-677250c7ecdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:41.252423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444246a0-a5db-48ac-9836-677250c7ecdc admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:41.256134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4d08b-61f1-4941-92c7-62f625a2adc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:41.257804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4d08b-61f1-4941-92c7-62f625a2adc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.258562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4d08b-61f1-4941-92c7-62f625a2adc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.259177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4d08b-61f1-4941-92c7-62f625a2adc1 admin admin] PATH_INFO: `/v3/projects/724d5517193d424e9a5a6ab9b853df5d/users/0506ea0497344111ab346c33e403c321/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.287742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 821/1624] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:41.289535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444246a0-a5db-48ac-9836-677250c7ecdc admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.301903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4d08b-61f1-4941-92c7-62f625a2adc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=724d5517193d424e9a5a6ab9b853df5d, user_id=0506ea0497344111ab346c33e403c321, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:41.308092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4d08b-61f1-4941-92c7-62f625a2adc1 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:41.308555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:41.315347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444246a0-a5db-48ac-9836-677250c7ecdc admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:41.326152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bbcdc-8734-4d70-950d-7d0bdc1ea61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:41.330160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bbcdc-8734-4d70-950d-7d0bdc1ea61f admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.333204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bbcdc-8734-4d70-950d-7d0bdc1ea61f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.333550 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bbcdc-8734-4d70-950d-7d0bdc1ea61f admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.334352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bbcdc-8734-4d70-950d-7d0bdc1ea61f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:41.335148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 804/1625] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:41 2022] PUT /identity/v3/projects/724d5517193d424e9a5a6ab9b853df5d/users/0506ea0497344111ab346c33e403c321/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:41.339088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4d08b-61f1-4941-92c7-62f625a2adc1 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.339405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bbcdc-8734-4d70-950d-7d0bdc1ea61f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:41.364314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 822/1626] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:41.365294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bbcdc-8734-4d70-950d-7d0bdc1ea61f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.386572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bbcdc-8734-4d70-950d-7d0bdc1ea61f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:41.486916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4d08b-61f1-4941-92c7-62f625a2adc1 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:41.501099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f70b3-06de-4060-a24a-e34f4893c535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:41.502213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f70b3-06de-4060-a24a-e34f4893c535 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.502509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f70b3-06de-4060-a24a-e34f4893c535 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.502791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f70b3-06de-4060-a24a-e34f4893c535 admin admin] PATH_INFO: `/v3/projects/d6d92b89f2ce4c189004c84a5b55811b/users/ddfb390192de436ab832a184fe4fa455/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.517798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39148ded-99bd-400c-88e6-ae506df8a369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:41.520468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39148ded-99bd-400c-88e6-ae506df8a369 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.520833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39148ded-99bd-400c-88e6-ae506df8a369 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.523138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39148ded-99bd-400c-88e6-ae506df8a369 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.528797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39148ded-99bd-400c-88e6-ae506df8a369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:41.535756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39148ded-99bd-400c-88e6-ae506df8a369 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:41.539582 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f70b3-06de-4060-a24a-e34f4893c535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6d92b89f2ce4c189004c84a5b55811b, user_id=ddfb390192de436ab832a184fe4fa455, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:41.542404 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f70b3-06de-4060-a24a-e34f4893c535 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:41.570155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 823/1627] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:41.572026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39148ded-99bd-400c-88e6-ae506df8a369 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.575847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 805/1628] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:41 2022] PUT /identity/v3/projects/d6d92b89f2ce4c189004c84a5b55811b/users/ddfb390192de436ab832a184fe4fa455/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:41.580545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f70b3-06de-4060-a24a-e34f4893c535 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.611999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39148ded-99bd-400c-88e6-ae506df8a369 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:41.653467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f70b3-06de-4060-a24a-e34f4893c535 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:41.659418 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad4618-73a8-4f3c-a78e-8fe9ce8f910a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:41.660337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad4618-73a8-4f3c-a78e-8fe9ce8f910a admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.660663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad4618-73a8-4f3c-a78e-8fe9ce8f910a admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.661009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad4618-73a8-4f3c-a78e-8fe9ce8f910a admin admin] PATH_INFO: `/v3/projects/724d5517193d424e9a5a6ab9b853df5d/users/0506ea0497344111ab346c33e403c321/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.663545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad4618-73a8-4f3c-a78e-8fe9ce8f910a admin admin] RBAC: Authorizing `identity:create_grant(project_id=724d5517193d424e9a5a6ab9b853df5d, user_id=0506ea0497344111ab346c33e403c321, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:41.666380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad4618-73a8-4f3c-a78e-8fe9ce8f910a admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:41.678911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 806/1629] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:41 2022] PUT /identity/v3/projects/724d5517193d424e9a5a6ab9b853df5d/users/0506ea0497344111ab346c33e403c321/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:41.690236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad4618-73a8-4f3c-a78e-8fe9ce8f910a admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.721200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:41.743526 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635d0ef-f71c-40a4-af78-fa3f07426d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:41.747435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635d0ef-f71c-40a4-af78-fa3f07426d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.749063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635d0ef-f71c-40a4-af78-fa3f07426d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.749578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635d0ef-f71c-40a4-af78-fa3f07426d47 admin admin] PATH_INFO: `/v3/projects/36d97558e37e422e87c96c6e21daec56/users/0ec61b626a2c445696cf70cab6f3c7dc/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.791622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad4618-73a8-4f3c-a78e-8fe9ce8f910a admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:41.797020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df966aff-bf95-4ee4-b3df-9d4d0a41c544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:41.798590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df966aff-bf95-4ee4-b3df-9d4d0a41c544 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.798942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df966aff-bf95-4ee4-b3df-9d4d0a41c544 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.799219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df966aff-bf95-4ee4-b3df-9d4d0a41c544 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.799976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df966aff-bf95-4ee4-b3df-9d4d0a41c544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:41.802087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df966aff-bf95-4ee4-b3df-9d4d0a41c544 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:41.809255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635d0ef-f71c-40a4-af78-fa3f07426d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d97558e37e422e87c96c6e21daec56, user_id=0ec61b626a2c445696cf70cab6f3c7dc, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:41.819057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 807/1630] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:41.820623 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385fda2-316a-4887-8e3f-024837c495c2 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:41.820969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385fda2-316a-4887-8e3f-024837c495c2 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:41.821255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385fda2-316a-4887-8e3f-024837c495c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:41.823434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635d0ef-f71c-40a4-af78-fa3f07426d47 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:41.862794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 824/1631] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:41 2022] PUT /identity/v3/projects/36d97558e37e422e87c96c6e21daec56/users/0ec61b626a2c445696cf70cab6f3c7dc/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:41.863935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635d0ef-f71c-40a4-af78-fa3f07426d47 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:41.874470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-2385fda2-316a-4887-8e3f-024837c495c2 None None] MFA Rules not processed for user `0506ea0497344111ab346c33e403c321`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:41.876274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2385fda2-316a-4887-8e3f-024837c495c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:41.997303 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635d0ef-f71c-40a4-af78-fa3f07426d47 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:42.013614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510826fa-41b2-4555-8038-ad680a54b1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:42.016856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510826fa-41b2-4555-8038-ad680a54b1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:42.017155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510826fa-41b2-4555-8038-ad680a54b1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:42.017500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510826fa-41b2-4555-8038-ad680a54b1b1 admin admin] PATH_INFO: `/v3/projects/d6d92b89f2ce4c189004c84a5b55811b/users/ddfb390192de436ab832a184fe4fa455/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:42.020847 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510826fa-41b2-4555-8038-ad680a54b1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6d92b89f2ce4c189004c84a5b55811b, user_id=ddfb390192de436ab832a184fe4fa455, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:42.024399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510826fa-41b2-4555-8038-ad680a54b1b1 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:42.053081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 825/1632] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:41 2022] PUT /identity/v3/projects/d6d92b89f2ce4c189004c84a5b55811b/users/ddfb390192de436ab832a184fe4fa455/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:42.053081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510826fa-41b2-4555-8038-ad680a54b1b1 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:42.075465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:42.082070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 808/1633] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:42.083856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a1e1e-76f5-4b85-8cc6-370c720f2f29 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:42.084390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a1e1e-76f5-4b85-8cc6-370c720f2f29 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:42.084948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a1e1e-76f5-4b85-8cc6-370c720f2f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:42.171336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-b22a1e1e-76f5-4b85-8cc6-370c720f2f29 None None] MFA Rules not processed for user `ddfb390192de436ab832a184fe4fa455`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:42.172693 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b22a1e1e-76f5-4b85-8cc6-370c720f2f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:42.322296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510826fa-41b2-4555-8038-ad680a54b1b1 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:42.322296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c82081-9841-4700-a5b5-1570dc39a422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:42.322296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c82081-9841-4700-a5b5-1570dc39a422 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:42.322296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c82081-9841-4700-a5b5-1570dc39a422 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:42.325520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c82081-9841-4700-a5b5-1570dc39a422 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:42.325520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c82081-9841-4700-a5b5-1570dc39a422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:42.325520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c82081-9841-4700-a5b5-1570dc39a422 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:42.325520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 826/1634] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:42.325520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c82081-9841-4700-a5b5-1570dc39a422 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:42.403226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 809/1635] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:42.404247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c82081-9841-4700-a5b5-1570dc39a422 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:42.405681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a1e1e-76f5-4b85-8cc6-370c720f2f29 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:42.410934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a1d52-370a-4f41-929f-8635e3cb9d27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:42.411679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a1d52-370a-4f41-929f-8635e3cb9d27 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:42.412033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a1d52-370a-4f41-929f-8635e3cb9d27 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:42.412318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a1d52-370a-4f41-929f-8635e3cb9d27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:42.420016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a1e1e-76f5-4b85-8cc6-370c720f2f29 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:42.427542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc88d6d-6ea4-4fa7-936d-1c43d9197ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:42.429364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc88d6d-6ea4-4fa7-936d-1c43d9197ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:42.430413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc88d6d-6ea4-4fa7-936d-1c43d9197ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:42.430413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc88d6d-6ea4-4fa7-936d-1c43d9197ef2 admin admin] PATH_INFO: `/v3/projects/36d97558e37e422e87c96c6e21daec56/users/0ec61b626a2c445696cf70cab6f3c7dc/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:42.433823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc88d6d-6ea4-4fa7-936d-1c43d9197ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d97558e37e422e87c96c6e21daec56, user_id=0ec61b626a2c445696cf70cab6f3c7dc, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:42.440189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc88d6d-6ea4-4fa7-936d-1c43d9197ef2 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:42.455718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:42.485708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 810/1636] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:42 2022] PUT /identity/v3/projects/36d97558e37e422e87c96c6e21daec56/users/0ec61b626a2c445696cf70cab6f3c7dc/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:42.522210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc88d6d-6ea4-4fa7-936d-1c43d9197ef2 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:42.585686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a1d52-370a-4f41-929f-8635e3cb9d27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:42.588094 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a1d52-370a-4f41-929f-8635e3cb9d27 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:42.603124 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 827/1637] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:42 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:42.607673 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4a488-dade-4ff3-8226-321e4d97f910 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:42.608096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4a488-dade-4ff3-8226-321e4d97f910 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:42.608623 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4a488-dade-4ff3-8226-321e4d97f910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:42.672961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-79e4a488-dade-4ff3-8226-321e4d97f910 None None] MFA Rules not processed for user `0ec61b626a2c445696cf70cab6f3c7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:42.674492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-79e4a488-dade-4ff3-8226-321e4d97f910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:42.759450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:42.772838 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc88d6d-6ea4-4fa7-936d-1c43d9197ef2 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:42.787927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71488e86-03e6-4891-8e6f-7eb384ca609a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:42.788961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71488e86-03e6-4891-8e6f-7eb384ca609a service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:42.789502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71488e86-03e6-4891-8e6f-7eb384ca609a service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:42.790024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71488e86-03e6-4891-8e6f-7eb384ca609a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:42.824442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 828/1638] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:42 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:42.896529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4a488-dade-4ff3-8226-321e4d97f910 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:43.009434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71488e86-03e6-4891-8e6f-7eb384ca609a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:43.010815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71488e86-03e6-4891-8e6f-7eb384ca609a service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:43.015295 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4a488-dade-4ff3-8226-321e4d97f910 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:43.020413 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:43.026755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:43.027131 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:43.027309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 811/1639] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:43.027600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] PATH_INFO: `/v3/users/a18555c9510c453280d50712f2e22bd0` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:43.028356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71488e86-03e6-4891-8e6f-7eb384ca609a service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:43.029360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a18555c9510c453280d50712f2e22bd0)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:43.030496 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: WARNING py.warnings [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:45:43.030496 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: warnings.warn(msg) Jul 25 08:45:43.030496 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: Jul 25 08:45:43.033052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:43.037443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71488e86-03e6-4891-8e6f-7eb384ca609a service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:43.062097 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a8bac-b18e-4f84-8233-e0a189d5827b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:43.068635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a8bac-b18e-4f84-8233-e0a189d5827b service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:43.069085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a8bac-b18e-4f84-8233-e0a189d5827b service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:43.069485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a8bac-b18e-4f84-8233-e0a189d5827b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:43.110912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a8bac-b18e-4f84-8233-e0a189d5827b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:43.119907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a8bac-b18e-4f84-8233-e0a189d5827b service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:43.182504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 812/1640] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:43 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:43.223976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a18555c9510c453280d50712f2e22bd0'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:43.225034 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a18555c9510c453280d50712f2e22bd0'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:43.250334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a18555c9510c453280d50712f2e22bd0'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:43.302536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a18555c9510c453280d50712f2e22bd0'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:43.355221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a18555c9510c453280d50712f2e22bd0'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:43.376672 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 829/1641] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:45:42 2022] DELETE /identity/v3/users/a18555c9510c453280d50712f2e22bd0 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:43.381820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb598f74-96c8-42f5-b779-d8bc80b07f9a None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:43.382270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb598f74-96c8-42f5-b779-d8bc80b07f9a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:43.382594 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb598f74-96c8-42f5-b779-d8bc80b07f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:43.425707 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-eb598f74-96c8-42f5-b779-d8bc80b07f9a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:43.428908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:43.431052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:43.442645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-eb598f74-96c8-42f5-b779-d8bc80b07f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:43.577956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc7104-4b17-4840-aa61-979496edb28b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:43.583311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877918cd-ded5-4004-bbdb-525611dcbdbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:43.584302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877918cd-ded5-4004-bbdb-525611dcbdbc service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:43.584642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877918cd-ded5-4004-bbdb-525611dcbdbc service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:43.585468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877918cd-ded5-4004-bbdb-525611dcbdbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:43.587701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-877918cd-ded5-4004-bbdb-525611dcbdbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:43.602407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:43.612651 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 813/1642] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:43.616318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb598f74-96c8-42f5-b779-d8bc80b07f9a None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:43.617445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-eb598f74-96c8-42f5-b779-d8bc80b07f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:43.654279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb598f74-96c8-42f5-b779-d8bc80b07f9a None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:43.671625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab45aec-d464-48ca-9ce1-bb8cb9dcf7d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:43.674663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab45aec-d464-48ca-9ce1-bb8cb9dcf7d3 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:43.675141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab45aec-d464-48ca-9ce1-bb8cb9dcf7d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:43.675918 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab45aec-d464-48ca-9ce1-bb8cb9dcf7d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:43.682295 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-dab45aec-d464-48ca-9ce1-bb8cb9dcf7d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:43.743650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877918cd-ded5-4004-bbdb-525611dcbdbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:43.745623 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877918cd-ded5-4004-bbdb-525611dcbdbc service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:43.759689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 830/1643] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:43.761944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877918cd-ded5-4004-bbdb-525611dcbdbc service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:43.764226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-877918cd-ded5-4004-bbdb-525611dcbdbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:43.822817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:43.905109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab45aec-d464-48ca-9ce1-bb8cb9dcf7d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:43.910019 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab45aec-d464-48ca-9ce1-bb8cb9dcf7d3 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:43.913492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877918cd-ded5-4004-bbdb-525611dcbdbc service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:43.926762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99ecbc-2ad6-494b-9312-f0010a11f2e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:43.927572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99ecbc-2ad6-494b-9312-f0010a11f2e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:43.928278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99ecbc-2ad6-494b-9312-f0010a11f2e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:43.928962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99ecbc-2ad6-494b-9312-f0010a11f2e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:43.936304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 814/1644] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:43 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:43.966920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99ecbc-2ad6-494b-9312-f0010a11f2e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:43.976254 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99ecbc-2ad6-494b-9312-f0010a11f2e6 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:44.026788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 831/1645] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:44.095440 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863b95-4df1-425e-acd3-3b790ed1037c None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:44.095905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863b95-4df1-425e-acd3-3b790ed1037c None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:44.096315 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863b95-4df1-425e-acd3-3b790ed1037c None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:44.098593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 815/1646] 10.222.0.52 () {54 vars in 962 bytes} [Mon Jul 25 08:45:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:45:44.142021 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263cb449-2b27-4f9a-9e49-df1664bf9522 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:44.142318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263cb449-2b27-4f9a-9e49-df1664bf9522 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:44.142623 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263cb449-2b27-4f9a-9e49-df1664bf9522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:44.248103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-263cb449-2b27-4f9a-9e49-df1664bf9522 None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:44.252898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-263cb449-2b27-4f9a-9e49-df1664bf9522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:44.480327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 832/1647] 10.222.0.52 () {58 vars in 1059 bytes} [Mon Jul 25 08:45:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:44.491408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c0084-f8f7-47b8-8922-96d42a75a9fc None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:44.491916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c0084-f8f7-47b8-8922-96d42a75a9fc None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:44.492406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c0084-f8f7-47b8-8922-96d42a75a9fc None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:44.494361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 816/1648] 10.222.0.52 () {54 vars in 912 bytes} [Mon Jul 25 08:45:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:45:44.503253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1335e6-3b59-485c-845f-fb79a3c7e26a None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:44.503549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1335e6-3b59-485c-845f-fb79a3c7e26a None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:44.503834 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1335e6-3b59-485c-845f-fb79a3c7e26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:44.551879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-fa1335e6-3b59-485c-845f-fb79a3c7e26a None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:44.573487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fa1335e6-3b59-485c-845f-fb79a3c7e26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:44.607567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 833/1649] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:45:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:44.620645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c0084-f8f7-47b8-8922-96d42a75a9fc None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:44.644306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c0084-f8f7-47b8-8922-96d42a75a9fc None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:44.652329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede157e6-df3c-4a7c-a500-ce9c11f1ad57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:44.653319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede157e6-df3c-4a7c-a500-ce9c11f1ad57 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:44.653809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede157e6-df3c-4a7c-a500-ce9c11f1ad57 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:44.654281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede157e6-df3c-4a7c-a500-ce9c11f1ad57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:44.690773 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede157e6-df3c-4a7c-a500-ce9c11f1ad57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:44.692512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede157e6-df3c-4a7c-a500-ce9c11f1ad57 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:44.741477 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 817/1650] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:46.395252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1335e6-3b59-485c-845f-fb79a3c7e26a None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:46.396785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fa1335e6-3b59-485c-845f-fb79a3c7e26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:46.438609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1335e6-3b59-485c-845f-fb79a3c7e26a None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:46.453243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:46.454194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:46.454491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:46.454770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97 admin admin] PATH_INFO: `/v3/projects/662a6c1838d64a028ee6fd1de8d58c2a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:46.460512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=662a6c1838d64a028ee6fd1de8d58c2a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:46.463287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:46.524088 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:46.585972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '662a6c1838d64a028ee6fd1de8d58c2a', 'request_id': 'req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:46.606637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '662a6c1838d64a028ee6fd1de8d58c2a', 'request_id': 'req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:46.617984 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '662a6c1838d64a028ee6fd1de8d58c2a', 'request_id': 'req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:46.643887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97 admin admin] The token cache is being invalidate because project 662a6c1838d64a028ee6fd1de8d58c2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:45:46.644071 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:46.653172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8fd28328-7fa8-468c-a839-e1b0bd5b5a97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:46.655236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 834/1651] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:45:46 2022] DELETE /identity/v3/projects/662a6c1838d64a028ee6fd1de8d58c2a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:46.659135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede157e6-df3c-4a7c-a500-ce9c11f1ad57 service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:46.672014 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede157e6-df3c-4a7c-a500-ce9c11f1ad57 service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:46.817976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d704f-0023-4045-91be-29404e8238a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:46.820064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d704f-0023-4045-91be-29404e8238a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:46.820606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d704f-0023-4045-91be-29404e8238a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:46.821754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d704f-0023-4045-91be-29404e8238a3 admin admin] PATH_INFO: `/v3/users/0506ea0497344111ab346c33e403c321` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:46.825431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d704f-0023-4045-91be-29404e8238a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0506ea0497344111ab346c33e403c321)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:46.832216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d704f-0023-4045-91be-29404e8238a3 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:46.969773 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ce4d704f-0023-4045-91be-29404e8238a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0506ea0497344111ab346c33e403c321'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:47.000491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ce4d704f-0023-4045-91be-29404e8238a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0506ea0497344111ab346c33e403c321'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:47.037122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ce4d704f-0023-4045-91be-29404e8238a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0506ea0497344111ab346c33e403c321'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:47.064986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ce4d704f-0023-4045-91be-29404e8238a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0506ea0497344111ab346c33e403c321'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:47.068854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ce4d704f-0023-4045-91be-29404e8238a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0506ea0497344111ab346c33e403c321'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:47.087756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 818/1652] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:45:46 2022] DELETE /identity/v3/users/0506ea0497344111ab346c33e403c321 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:47.377259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fcf9a-804e-45a1-bc2a-2b8589d72db2 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:47.377782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fcf9a-804e-45a1-bc2a-2b8589d72db2 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:47.378068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fcf9a-804e-45a1-bc2a-2b8589d72db2 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:47.381570 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 835/1653] 10.222.0.52 () {54 vars in 962 bytes} [Mon Jul 25 08:45:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:45:47.393262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725fde8-472a-4ee8-b7b8-0e2a7a4cae66 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:47.393768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725fde8-472a-4ee8-b7b8-0e2a7a4cae66 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:47.394188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725fde8-472a-4ee8-b7b8-0e2a7a4cae66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:47.437471 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-1725fde8-472a-4ee8-b7b8-0e2a7a4cae66 None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:47.443182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1725fde8-472a-4ee8-b7b8-0e2a7a4cae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:47.571428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:47.675705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 819/1654] 10.222.0.52 () {58 vars in 1059 bytes} [Mon Jul 25 08:45:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:47.684067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8746f-e20f-4cac-a3ed-4a68d5a2ab0c None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:47.684952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8746f-e20f-4cac-a3ed-4a68d5a2ab0c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:47.685236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8746f-e20f-4cac-a3ed-4a68d5a2ab0c None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:47.687024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 836/1655] 10.222.0.52 () {54 vars in 912 bytes} [Mon Jul 25 08:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:45:47.701006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8e515-189b-46de-8d49-c9cb7951f7e7 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:47.701297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8e515-189b-46de-8d49-c9cb7951f7e7 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:47.701582 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8e515-189b-46de-8d49-c9cb7951f7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:47.739817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-cbd8e515-189b-46de-8d49-c9cb7951f7e7 None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:47.743602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-cbd8e515-189b-46de-8d49-c9cb7951f7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:47.783020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 820/1656] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:45:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:47.920824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8746f-e20f-4cac-a3ed-4a68d5a2ab0c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:47.922938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-16f8746f-e20f-4cac-a3ed-4a68d5a2ab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:48.065712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8746f-e20f-4cac-a3ed-4a68d5a2ab0c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:48.087636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a0046-4f0e-4888-9075-ca67412c2938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:48.089420 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a0046-4f0e-4888-9075-ca67412c2938 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:48.089995 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a0046-4f0e-4888-9075-ca67412c2938 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:48.090814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a0046-4f0e-4888-9075-ca67412c2938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:48.117341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a0046-4f0e-4888-9075-ca67412c2938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:48.120526 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a0046-4f0e-4888-9075-ca67412c2938 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:48.159853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 837/1657] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:48.384960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8e515-189b-46de-8d49-c9cb7951f7e7 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:48.386980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-cbd8e515-189b-46de-8d49-c9cb7951f7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:48.523789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8e515-189b-46de-8d49-c9cb7951f7e7 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:48.529609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:48.530786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:48.531070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:48.531416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] PATH_INFO: `/v3/projects/724d5517193d424e9a5a6ab9b853df5d` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:48.541793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=724d5517193d424e9a5a6ab9b853df5d)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:48.543246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: WARNING py.warnings [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 08:45:48.543246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: warnings.warn(msg) Jul 25 08:45:48.543246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: Jul 25 08:45:48.545053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:48.643987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '724d5517193d424e9a5a6ab9b853df5d', 'request_id': 'req-ed3b8e59-e482-48fb-85c5-81a0f7841abb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:48.654822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '724d5517193d424e9a5a6ab9b853df5d', 'request_id': 'req-ed3b8e59-e482-48fb-85c5-81a0f7841abb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:48.666050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '724d5517193d424e9a5a6ab9b853df5d', 'request_id': 'req-ed3b8e59-e482-48fb-85c5-81a0f7841abb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:48.692663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] The token cache is being invalidate because project 724d5517193d424e9a5a6ab9b853df5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:45:48.692663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:48.693919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ed3b8e59-e482-48fb-85c5-81a0f7841abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:45:48.696166 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 821/1658] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:45:48 2022] DELETE /identity/v3/projects/724d5517193d424e9a5a6ab9b853df5d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:48.701138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede744a-965b-454a-b567-406708263ba5 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:48.701643 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede744a-965b-454a-b567-406708263ba5 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:48.701817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede744a-965b-454a-b567-406708263ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:48.777780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-bede744a-965b-454a-b567-406708263ba5 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:48.808037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-bede744a-965b-454a-b567-406708263ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:48.988744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 838/1659] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:48.997559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4aa900-070e-4512-bf98-9083d2ca65a9 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:48.997921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4aa900-070e-4512-bf98-9083d2ca65a9 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:48.998285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4aa900-070e-4512-bf98-9083d2ca65a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:49.040484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-dd4aa900-070e-4512-bf98-9083d2ca65a9 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:49.041865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-dd4aa900-070e-4512-bf98-9083d2ca65a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:49.062559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 822/1660] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:49.064323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede744a-965b-454a-b567-406708263ba5 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:49.085194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede744a-965b-454a-b567-406708263ba5 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:49.098278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e70bf1-9e0d-4203-97ca-18a7b7c0db52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:49.099402 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e70bf1-9e0d-4203-97ca-18a7b7c0db52 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:49.099939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e70bf1-9e0d-4203-97ca-18a7b7c0db52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:49.100397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e70bf1-9e0d-4203-97ca-18a7b7c0db52 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:49.101337 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e70bf1-9e0d-4203-97ca-18a7b7c0db52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:45:49.102747 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e70bf1-9e0d-4203-97ca-18a7b7c0db52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:49.118763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e70bf1-9e0d-4203-97ca-18a7b7c0db52 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:49.144004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 839/1661] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:45:49.148787 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4aa900-070e-4512-bf98-9083d2ca65a9 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:49.175492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4aa900-070e-4512-bf98-9083d2ca65a9 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:49.194661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd2d4d-fb80-4124-92ea-fec6146f9286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:49.195355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd2d4d-fb80-4124-92ea-fec6146f9286 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:49.195640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd2d4d-fb80-4124-92ea-fec6146f9286 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:49.196187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd2d4d-fb80-4124-92ea-fec6146f9286 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:49.197579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd2d4d-fb80-4124-92ea-fec6146f9286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:49.201981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd2d4d-fb80-4124-92ea-fec6146f9286 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:49.282390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 823/1662] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:49 2022] POST /identity/v3/projects => generated 405 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:45:49.282941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e70bf1-9e0d-4203-97ca-18a7b7c0db52 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:49.299693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e70bf1-9e0d-4203-97ca-18a7b7c0db52 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:49.377310 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:49.427755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44268a1-c26e-46d1-9b16-798812876743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:49.428466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44268a1-c26e-46d1-9b16-798812876743 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:49.428870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44268a1-c26e-46d1-9b16-798812876743 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:49.429401 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44268a1-c26e-46d1-9b16-798812876743 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:49.429902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44268a1-c26e-46d1-9b16-798812876743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:49.436040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44268a1-c26e-46d1-9b16-798812876743 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:49.486360 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd2d4d-fb80-4124-92ea-fec6146f9286 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:49.492986 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 840/1663] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:49 2022] POST /identity/v3/users => generated 303 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:45:49.517099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd2d4d-fb80-4124-92ea-fec6146f9286 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:49.532997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4327d-75b8-400b-8671-208cecb63267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:49.533706 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4327d-75b8-400b-8671-208cecb63267 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:49.533996 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4327d-75b8-400b-8671-208cecb63267 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:49.534280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4327d-75b8-400b-8671-208cecb63267 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:49.535036 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4327d-75b8-400b-8671-208cecb63267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:49.536720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4327d-75b8-400b-8671-208cecb63267 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:49.580013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 824/1664] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:49.585619 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44268a1-c26e-46d1-9b16-798812876743 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:49.620480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44268a1-c26e-46d1-9b16-798812876743 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:49.660548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d0a11-6f02-491e-86ea-3592fb41956e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:49.661157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d0a11-6f02-491e-86ea-3592fb41956e admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:49.661482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d0a11-6f02-491e-86ea-3592fb41956e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:49.661878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d0a11-6f02-491e-86ea-3592fb41956e admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:49.662694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d0a11-6f02-491e-86ea-3592fb41956e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:49.666769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d0a11-6f02-491e-86ea-3592fb41956e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:49.697561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 841/1665] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:49.698724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4327d-75b8-400b-8671-208cecb63267 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:49.711295 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4327d-75b8-400b-8671-208cecb63267 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:49.726414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115d1cc-ed71-4756-988f-aadbb453fbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:49.727505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115d1cc-ed71-4756-988f-aadbb453fbf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:49.727949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115d1cc-ed71-4756-988f-aadbb453fbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:49.728398 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115d1cc-ed71-4756-988f-aadbb453fbf4 admin admin] PATH_INFO: `/v3/projects/3c81303b75d642c69cf80e9ecfb88724/users/5dfe337446274a0d950ce9bf2acdb09a/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:49.763655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:49.772659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115d1cc-ed71-4756-988f-aadbb453fbf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c81303b75d642c69cf80e9ecfb88724, user_id=5dfe337446274a0d950ce9bf2acdb09a, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:49.785088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115d1cc-ed71-4756-988f-aadbb453fbf4 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:49.826309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 825/1666] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:49 2022] PUT /identity/v3/projects/3c81303b75d642c69cf80e9ecfb88724/users/5dfe337446274a0d950ce9bf2acdb09a/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:49.834170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dc9d8-2f7e-4d55-a4d0-5ba46aa25caf None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:49.835902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dc9d8-2f7e-4d55-a4d0-5ba46aa25caf None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:49.836193 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dc9d8-2f7e-4d55-a4d0-5ba46aa25caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:49.876041 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-405dc9d8-2f7e-4d55-a4d0-5ba46aa25caf None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:49.879297 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-405dc9d8-2f7e-4d55-a4d0-5ba46aa25caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:50.091949 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 842/1667] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:50.157297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115d1cc-ed71-4756-988f-aadbb453fbf4 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:50.159497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7115d1cc-ed71-4756-988f-aadbb453fbf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:50.300623 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115d1cc-ed71-4756-988f-aadbb453fbf4 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:50.309653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316fea6-04b9-45d2-8076-7ed8ffbcd386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:50.310538 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316fea6-04b9-45d2-8076-7ed8ffbcd386 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:50.310980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316fea6-04b9-45d2-8076-7ed8ffbcd386 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:50.311394 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316fea6-04b9-45d2-8076-7ed8ffbcd386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:50.325358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316fea6-04b9-45d2-8076-7ed8ffbcd386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:50.327367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316fea6-04b9-45d2-8076-7ed8ffbcd386 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:50.339861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 826/1668] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:51.928550 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d8389-a941-45a5-8d16-5b650b7a1365 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:51.929156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d8389-a941-45a5-8d16-5b650b7a1365 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:51.929815 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d8389-a941-45a5-8d16-5b650b7a1365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:51.999076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c74d8389-a941-45a5-8d16-5b650b7a1365 None None] MFA Rules not processed for user `718ca333456f4de5b98f191ab40cca0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:52.001348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c74d8389-a941-45a5-8d16-5b650b7a1365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:52.133375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 843/1669] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:52.142412 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316fea6-04b9-45d2-8076-7ed8ffbcd386 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:52.144410 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-6316fea6-04b9-45d2-8076-7ed8ffbcd386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:52.294617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf10dc-5614-411a-8fb2-db374027ac19 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:52.294914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf10dc-5614-411a-8fb2-db374027ac19 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:52.295203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf10dc-5614-411a-8fb2-db374027ac19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:52.327704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316fea6-04b9-45d2-8076-7ed8ffbcd386 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:52.359469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f420f2-15a4-472a-a334-5eeec112ee05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:52.362432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f420f2-15a4-472a-a334-5eeec112ee05 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:52.364165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a8bf10dc-5614-411a-8fb2-db374027ac19 None None] MFA Rules not processed for user `238f78244df546e5a6fc1a4cd1b73288`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:52.364751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f420f2-15a4-472a-a334-5eeec112ee05 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:52.365247 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f420f2-15a4-472a-a334-5eeec112ee05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:52.366276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a8bf10dc-5614-411a-8fb2-db374027ac19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:52.382305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f420f2-15a4-472a-a334-5eeec112ee05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:52.384260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f420f2-15a4-472a-a334-5eeec112ee05 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:52.412560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 827/1670] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:52.463922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:52.521649 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 844/1671] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:52.525103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc027b-ce6c-49c5-aba2-bac08720a1e6 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:52.525493 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc027b-ce6c-49c5-aba2-bac08720a1e6 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:52.525792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc027b-ce6c-49c5-aba2-bac08720a1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:52.571698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-9bbc027b-ce6c-49c5-aba2-bac08720a1e6 None None] MFA Rules not processed for user `238f78244df546e5a6fc1a4cd1b73288`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:52.575032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-9bbc027b-ce6c-49c5-aba2-bac08720a1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:52.592953 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 828/1672] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:52.601641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf10dc-5614-411a-8fb2-db374027ac19 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:52.603093 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a8bf10dc-5614-411a-8fb2-db374027ac19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:52.642616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf10dc-5614-411a-8fb2-db374027ac19 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:52.673223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93790999-22dd-4db1-b0eb-f11c3e2afa04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:52.673938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93790999-22dd-4db1-b0eb-f11c3e2afa04 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:52.674227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93790999-22dd-4db1-b0eb-f11c3e2afa04 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:52.674509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93790999-22dd-4db1-b0eb-f11c3e2afa04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:52.694565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93790999-22dd-4db1-b0eb-f11c3e2afa04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:52.702108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93790999-22dd-4db1-b0eb-f11c3e2afa04 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:52.757072 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 845/1673] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:53.608794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f854e9-62f4-47ab-b3d5-8f1aed5069ba None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:53.609273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f854e9-62f4-47ab-b3d5-8f1aed5069ba None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:53.609367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f854e9-62f4-47ab-b3d5-8f1aed5069ba None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:53.611124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 829/1674] 10.222.0.52 () {54 vars in 912 bytes} [Mon Jul 25 08:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:45:53.617935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4ae7d-1518-4675-8880-e57734609992 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:53.618407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4ae7d-1518-4675-8880-e57734609992 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:53.618597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4ae7d-1518-4675-8880-e57734609992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:53.681200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-2ca4ae7d-1518-4675-8880-e57734609992 None None] MFA Rules not processed for user `62688bc3d8a449c5bc9e0d64cd2462b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:53.682803 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2ca4ae7d-1518-4675-8880-e57734609992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:53.711459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 846/1675] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:53.746817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f854e9-62f4-47ab-b3d5-8f1aed5069ba None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:53.749891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-43f854e9-62f4-47ab-b3d5-8f1aed5069ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:53.784518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f854e9-62f4-47ab-b3d5-8f1aed5069ba None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:53.791124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426f827-f1d4-4b13-9716-f2bf0f186bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:53.792082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426f827-f1d4-4b13-9716-f2bf0f186bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:53.792593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426f827-f1d4-4b13-9716-f2bf0f186bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:53.793075 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426f827-f1d4-4b13-9716-f2bf0f186bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:53.802556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426f827-f1d4-4b13-9716-f2bf0f186bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:53.806682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426f827-f1d4-4b13-9716-f2bf0f186bb0 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:53.843574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 830/1676] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:54.631129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885913eb-c645-4785-b51f-a90efec40da0 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:54.632195 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885913eb-c645-4785-b51f-a90efec40da0 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:54.632774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885913eb-c645-4785-b51f-a90efec40da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:54.706541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-885913eb-c645-4785-b51f-a90efec40da0 None None] MFA Rules not processed for user `5dfe337446274a0d950ce9bf2acdb09a`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:54.707849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-885913eb-c645-4785-b51f-a90efec40da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:54.915427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 847/1677] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:54 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:54.921206 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426f827-f1d4-4b13-9716-f2bf0f186bb0 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:54.942933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426f827-f1d4-4b13-9716-f2bf0f186bb0 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:54.952025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb5f96-7500-4591-8caa-8115cc75db23 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:54.952196 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb5f96-7500-4591-8caa-8115cc75db23 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:54.952370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb5f96-7500-4591-8caa-8115cc75db23 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:54.954203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 848/1678] 10.222.0.52 () {54 vars in 961 bytes} [Mon Jul 25 08:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:45:54.964123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f93c8-ff25-41fa-a74f-cdfb32fbe0e1 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:54.965553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f93c8-ff25-41fa-a74f-cdfb32fbe0e1 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:54.966076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f93c8-ff25-41fa-a74f-cdfb32fbe0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:54.970630 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec92b58-012d-4f40-8c38-f615d1ab8149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:54.971364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec92b58-012d-4f40-8c38-f615d1ab8149 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:54.971652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec92b58-012d-4f40-8c38-f615d1ab8149 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:54.971923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec92b58-012d-4f40-8c38-f615d1ab8149 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:54.972658 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec92b58-012d-4f40-8c38-f615d1ab8149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:54.977694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec92b58-012d-4f40-8c38-f615d1ab8149 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:55.064680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-003f93c8-ff25-41fa-a74f-cdfb32fbe0e1 None None] MFA Rules not processed for user `62bf4ef4fce74f7b91e453f73430d9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:55.066152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-003f93c8-ff25-41fa-a74f-cdfb32fbe0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:55.106170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 831/1679] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:45:54 2022] POST /identity/v3/projects => generated 405 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:55.107188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec92b58-012d-4f40-8c38-f615d1ab8149 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:55.199335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:55.266179 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec92b58-012d-4f40-8c38-f615d1ab8149 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:55.276795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb3ba1-3223-4a90-b925-61aaa05f9b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:55.277593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb3ba1-3223-4a90-b925-61aaa05f9b36 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:55.278050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb3ba1-3223-4a90-b925-61aaa05f9b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:55.278329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb3ba1-3223-4a90-b925-61aaa05f9b36 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:55.279415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb3ba1-3223-4a90-b925-61aaa05f9b36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:55.285064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb3ba1-3223-4a90-b925-61aaa05f9b36 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:55.292348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 849/1680] 10.222.0.52 () {58 vars in 1058 bytes} [Mon Jul 25 08:45:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:45:55.293903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956628cf-341d-49b4-938e-da4636e6953e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:55.294299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956628cf-341d-49b4-938e-da4636e6953e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:55.295275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956628cf-341d-49b4-938e-da4636e6953e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:55.321278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-956628cf-341d-49b4-938e-da4636e6953e None None] MFA Rules not processed for user `238f78244df546e5a6fc1a4cd1b73288`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:55.324531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-956628cf-341d-49b4-938e-da4636e6953e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:55.344537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 832/1681] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:45:55 2022] POST /identity/v3/users => generated 309 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:45:55.346666 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb3ba1-3223-4a90-b925-61aaa05f9b36 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:55.407865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:55.452909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb3ba1-3223-4a90-b925-61aaa05f9b36 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:55.459712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ad6e8-265d-4115-b1fc-4a05e4d9ea7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:55.460433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ad6e8-265d-4115-b1fc-4a05e4d9ea7f service placement] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:55.460811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ad6e8-265d-4115-b1fc-4a05e4d9ea7f service placement] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:55.461095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ad6e8-265d-4115-b1fc-4a05e4d9ea7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:55.467918 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-094ad6e8-265d-4115-b1fc-4a05e4d9ea7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:55.481136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ad6e8-265d-4115-b1fc-4a05e4d9ea7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:55.483066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ad6e8-265d-4115-b1fc-4a05e4d9ea7f service placement] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:55.491352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 850/1682] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:55.494068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956628cf-341d-49b4-938e-da4636e6953e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:55.499852 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 833/1683] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:55.524481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956628cf-341d-49b4-938e-da4636e6953e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:55.533319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95183a5c-8790-4684-96a6-42593c524ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:55.535195 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95183a5c-8790-4684-96a6-42593c524ffc admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:55.537429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95183a5c-8790-4684-96a6-42593c524ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:55.538661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95183a5c-8790-4684-96a6-42593c524ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:55.539990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95183a5c-8790-4684-96a6-42593c524ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:55.560071 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95183a5c-8790-4684-96a6-42593c524ffc admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:55.639602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 851/1684] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:55.641717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ad6e8-265d-4115-b1fc-4a05e4d9ea7f service placement] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:55.673832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ad6e8-265d-4115-b1fc-4a05e4d9ea7f service placement] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:55.707717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcabc5f-9cdf-4d20-ac2c-3f403739a18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:55.708930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcabc5f-9cdf-4d20-ac2c-3f403739a18c admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:55.709214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcabc5f-9cdf-4d20-ac2c-3f403739a18c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:55.709509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcabc5f-9cdf-4d20-ac2c-3f403739a18c admin admin] PATH_INFO: `/v3/projects/3f38a23055a84d6fb72057044c938075/users/45493ff555104c65b3d9cac8679b173e/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:55.782124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcabc5f-9cdf-4d20-ac2c-3f403739a18c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f38a23055a84d6fb72057044c938075, user_id=45493ff555104c65b3d9cac8679b173e, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:55.785611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcabc5f-9cdf-4d20-ac2c-3f403739a18c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:55.878019 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 834/1685] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:55 2022] PUT /identity/v3/projects/3f38a23055a84d6fb72057044c938075/users/45493ff555104c65b3d9cac8679b173e/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:55.881905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95183a5c-8790-4684-96a6-42593c524ffc admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:55.909615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95183a5c-8790-4684-96a6-42593c524ffc admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:56.113236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d865dd-4e62-41ab-83c6-594e7088b4e8 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:56.113236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d865dd-4e62-41ab-83c6-594e7088b4e8 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:56.113236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d865dd-4e62-41ab-83c6-594e7088b4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:56.118877 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f9145-0858-4cf4-bb6a-1be020086c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:56.120356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f9145-0858-4cf4-bb6a-1be020086c23 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:56.121976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f9145-0858-4cf4-bb6a-1be020086c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:56.123943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f9145-0858-4cf4-bb6a-1be020086c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:56.125895 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f9145-0858-4cf4-bb6a-1be020086c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:56.133536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f9145-0858-4cf4-bb6a-1be020086c23 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:56.190464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 852/1686] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:45:56.196992 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f9145-0858-4cf4-bb6a-1be020086c23 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:56.225861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f9145-0858-4cf4-bb6a-1be020086c23 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:56.247777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd906c4-7626-4313-8524-c7fd401ef4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:56.249026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd906c4-7626-4313-8524-c7fd401ef4aa admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:56.251637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd906c4-7626-4313-8524-c7fd401ef4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:56.252154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd906c4-7626-4313-8524-c7fd401ef4aa admin admin] PATH_INFO: `/v3/projects/3f38a23055a84d6fb72057044c938075/users/45493ff555104c65b3d9cac8679b173e/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:56.258313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd906c4-7626-4313-8524-c7fd401ef4aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f38a23055a84d6fb72057044c938075, user_id=45493ff555104c65b3d9cac8679b173e, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:56.259121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-31d865dd-4e62-41ab-83c6-594e7088b4e8 None None] MFA Rules not processed for user `bc39f30010e94eb5862285347575f673`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:56.260662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-31d865dd-4e62-41ab-83c6-594e7088b4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:56.270460 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd906c4-7626-4313-8524-c7fd401ef4aa admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:56.363825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 853/1687] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:45:56 2022] PUT /identity/v3/projects/3f38a23055a84d6fb72057044c938075/users/45493ff555104c65b3d9cac8679b173e/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:45:56.582421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 835/1688] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:56.593050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2402f3-52f4-4180-b841-0beb28faad9e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:56.593453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2402f3-52f4-4180-b841-0beb28faad9e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:56.594398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2402f3-52f4-4180-b841-0beb28faad9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:56.651539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-2f2402f3-52f4-4180-b841-0beb28faad9e None None] MFA Rules not processed for user `bc39f30010e94eb5862285347575f673`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:56.658400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2f2402f3-52f4-4180-b841-0beb28faad9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:56.704951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27c72e-52ad-4da1-923a-156cbb2b102a None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:56.705256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27c72e-52ad-4da1-923a-156cbb2b102a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:56.705545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27c72e-52ad-4da1-923a-156cbb2b102a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:56.781860 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ff27c72e-52ad-4da1-923a-156cbb2b102a None None] MFA Rules not processed for user `8e682232b7374450a7265e092602a5a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:56.786561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ff27c72e-52ad-4da1-923a-156cbb2b102a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:56.851107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:56.874772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 854/1689] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:57.040622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2402f3-52f4-4180-b841-0beb28faad9e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:57.042126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 836/1690] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:57.238783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:45:57.325553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2402f3-52f4-4180-b841-0beb28faad9e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:57.333699 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6b439-45ce-4359-9325-3627896e2f7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:57.338439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6b439-45ce-4359-9325-3627896e2f7d service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:57.338553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6b439-45ce-4359-9325-3627896e2f7d service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:57.338840 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6b439-45ce-4359-9325-3627896e2f7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:57.369511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6b439-45ce-4359-9325-3627896e2f7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:57.371140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6b439-45ce-4359-9325-3627896e2f7d service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:57.393229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 855/1691] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:58.195745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0f0ce-3b55-402e-adce-730113bb5ea1 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:58.196690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0f0ce-3b55-402e-adce-730113bb5ea1 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:58.197455 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0f0ce-3b55-402e-adce-730113bb5ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:58.255767 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-13b0f0ce-3b55-402e-adce-730113bb5ea1 None None] MFA Rules not processed for user `45493ff555104c65b3d9cac8679b173e`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:58.257719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-13b0f0ce-3b55-402e-adce-730113bb5ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:58.429321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668cc9b-32d3-4a08-8135-87994d9d366e None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:58.430546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668cc9b-32d3-4a08-8135-87994d9d366e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:58.430977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668cc9b-32d3-4a08-8135-87994d9d366e None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:58.433503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 856/1692] 10.222.0.52 () {54 vars in 912 bytes} [Mon Jul 25 08:45:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:45:58.442320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50914b-e84a-4c05-b1b8-500c874f149b None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:58.442914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50914b-e84a-4c05-b1b8-500c874f149b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:58.443557 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50914b-e84a-4c05-b1b8-500c874f149b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:58.461397 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 837/1693] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:58 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:45:58.486207 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-6b50914b-e84a-4c05-b1b8-500c874f149b None None] MFA Rules not processed for user `62688bc3d8a449c5bc9e0d64cd2462b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:58.487790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6b50914b-e84a-4c05-b1b8-500c874f149b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:58.611763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b33c72-21f0-41c7-8f63-47ec6f86a2da None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:58.612181 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b33c72-21f0-41c7-8f63-47ec6f86a2da None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:58.612594 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b33c72-21f0-41c7-8f63-47ec6f86a2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:58.660090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 857/1694] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:45:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:45:58.662090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50914b-e84a-4c05-b1b8-500c874f149b None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:58.667301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-d0b33c72-21f0-41c7-8f63-47ec6f86a2da None None] MFA Rules not processed for user `718ca333456f4de5b98f191ab40cca0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:58.668504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d0b33c72-21f0-41c7-8f63-47ec6f86a2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:45:58.690973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50914b-e84a-4c05-b1b8-500c874f149b None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:58.701654 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c0001-6be5-44e1-97cb-e24824c71e02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:58.702325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c0001-6be5-44e1-97cb-e24824c71e02 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:58.703812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c0001-6be5-44e1-97cb-e24824c71e02 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:58.703930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c0001-6be5-44e1-97cb-e24824c71e02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:58.717369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c0001-6be5-44e1-97cb-e24824c71e02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:58.722750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c0001-6be5-44e1-97cb-e24824c71e02 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:58.740279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 858/1695] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:58.741508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c0001-6be5-44e1-97cb-e24824c71e02 service nova] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:45:58.754353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c0001-6be5-44e1-97cb-e24824c71e02 service nova] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:45:58.760606 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba36c7-81fb-4d0f-98dc-43ec06c60905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:45:58.761602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba36c7-81fb-4d0f-98dc-43ec06c60905 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:58.761837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba36c7-81fb-4d0f-98dc-43ec06c60905 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:58.762120 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba36c7-81fb-4d0f-98dc-43ec06c60905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:58.772375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba36c7-81fb-4d0f-98dc-43ec06c60905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:45:58.775788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba36c7-81fb-4d0f-98dc-43ec06c60905 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:45:58.804939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 859/1696] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:45:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:45:58.894485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 838/1697] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:45:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:45:59.727312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09b2d6-e39a-4f7f-9998-16b00390437f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:45:59.728271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09b2d6-e39a-4f7f-9998-16b00390437f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:45:59.728818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09b2d6-e39a-4f7f-9998-16b00390437f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:45:59.861750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7a09b2d6-e39a-4f7f-9998-16b00390437f None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:45:59.862996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7a09b2d6-e39a-4f7f-9998-16b00390437f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:00.048791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 860/1698] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:00.298254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b33c72-21f0-41c7-8f63-47ec6f86a2da None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:00.299808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d0b33c72-21f0-41c7-8f63-47ec6f86a2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:00.352474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afaf56-cffc-40a8-9239-19a20ed7b5fd None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:00.352786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afaf56-cffc-40a8-9239-19a20ed7b5fd None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:00.353353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afaf56-cffc-40a8-9239-19a20ed7b5fd None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:00.355197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 861/1699] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:46:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:46:00.388353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e7fa7-19c6-4e95-9d43-be2a00b52260 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:00.388669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e7fa7-19c6-4e95-9d43-be2a00b52260 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:00.388956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e7fa7-19c6-4e95-9d43-be2a00b52260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:00.430900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:00.438609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c93e7fa7-19c6-4e95-9d43-be2a00b52260 None None] MFA Rules not processed for user `62bf4ef4fce74f7b91e453f73430d9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:00.439855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c93e7fa7-19c6-4e95-9d43-be2a00b52260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:00.569734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:00.613778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b33c72-21f0-41c7-8f63-47ec6f86a2da None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:00.631466 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6822c28-fbd9-47ec-88dd-d3b2beaca06a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:00.632774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6822c28-fbd9-47ec-88dd-d3b2beaca06a service swift] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:00.633251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6822c28-fbd9-47ec-88dd-d3b2beaca06a service swift] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:00.633784 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6822c28-fbd9-47ec-88dd-d3b2beaca06a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:00.646544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6822c28-fbd9-47ec-88dd-d3b2beaca06a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:00.649613 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6822c28-fbd9-47ec-88dd-d3b2beaca06a service swift] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:00.689107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 839/1700] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:46:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:46:00.751966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 862/1701] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:46:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 25 08:46:00.776507 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6822c28-fbd9-47ec-88dd-d3b2beaca06a service swift] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:00.778235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a6822c28-fbd9-47ec-88dd-d3b2beaca06a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:00.820379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6822c28-fbd9-47ec-88dd-d3b2beaca06a service swift] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:00.836850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866c63a-6bb0-4b91-b2d7-f36e994cc5d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:00.837532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866c63a-6bb0-4b91-b2d7-f36e994cc5d7 service placement] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:00.837824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866c63a-6bb0-4b91-b2d7-f36e994cc5d7 service placement] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:00.838109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866c63a-6bb0-4b91-b2d7-f36e994cc5d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:00.886150 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866c63a-6bb0-4b91-b2d7-f36e994cc5d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:00.894147 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866c63a-6bb0-4b91-b2d7-f36e994cc5d7 service placement] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:00.925309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 840/1702] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:04.219481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e7fa7-19c6-4e95-9d43-be2a00b52260 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:04.441387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e7fa7-19c6-4e95-9d43-be2a00b52260 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:04.462533 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54f638-161c-4e5e-8c9c-8c51c5ed7886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:04.464506 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54f638-161c-4e5e-8c9c-8c51c5ed7886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:04.464999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54f638-161c-4e5e-8c9c-8c51c5ed7886 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:04.465437 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54f638-161c-4e5e-8c9c-8c51c5ed7886 admin admin] PATH_INFO: `/v3/users/5dfe337446274a0d950ce9bf2acdb09a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:04.472714 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54f638-161c-4e5e-8c9c-8c51c5ed7886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dfe337446274a0d950ce9bf2acdb09a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:04.480846 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54f638-161c-4e5e-8c9c-8c51c5ed7886 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:04.651473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-aa54f638-161c-4e5e-8c9c-8c51c5ed7886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dfe337446274a0d950ce9bf2acdb09a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:04.657113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-aa54f638-161c-4e5e-8c9c-8c51c5ed7886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dfe337446274a0d950ce9bf2acdb09a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:04.696786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-aa54f638-161c-4e5e-8c9c-8c51c5ed7886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dfe337446274a0d950ce9bf2acdb09a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:04.792017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-aa54f638-161c-4e5e-8c9c-8c51c5ed7886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dfe337446274a0d950ce9bf2acdb09a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:04.836261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-aa54f638-161c-4e5e-8c9c-8c51c5ed7886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dfe337446274a0d950ce9bf2acdb09a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:04.877394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 863/1703] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:04 2022] DELETE /identity/v3/users/5dfe337446274a0d950ce9bf2acdb09a => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:06.239639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866c63a-6bb0-4b91-b2d7-f36e994cc5d7 service placement] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:06.264922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866c63a-6bb0-4b91-b2d7-f36e994cc5d7 service placement] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:06.602139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:06.603008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:06.603348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:06.603626 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] PATH_INFO: `/v3/projects/3c81303b75d642c69cf80e9ecfb88724` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:06.613585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c81303b75d642c69cf80e9ecfb88724)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:06.619457 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:06.749386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a370efc-f723-4904-b5cb-fe4fcaee34d3 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:06.750935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a370efc-f723-4904-b5cb-fe4fcaee34d3 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:06.751427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a370efc-f723-4904-b5cb-fe4fcaee34d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:06.816599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c81303b75d642c69cf80e9ecfb88724', 'request_id': 'req-d30fffec-368b-43b1-885b-a342f50a3915'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:06.826267 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7a370efc-f723-4904-b5cb-fe4fcaee34d3 None None] MFA Rules not processed for user `62688bc3d8a449c5bc9e0d64cd2462b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:06.832476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7a370efc-f723-4904-b5cb-fe4fcaee34d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:06.838808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c81303b75d642c69cf80e9ecfb88724', 'request_id': 'req-d30fffec-368b-43b1-885b-a342f50a3915'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:06.861353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c81303b75d642c69cf80e9ecfb88724', 'request_id': 'req-d30fffec-368b-43b1-885b-a342f50a3915'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:06.896183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] The token cache is being invalidate because project 3c81303b75d642c69cf80e9ecfb88724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:06.896486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:06.897876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:06.899810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 841/1704] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:06 2022] DELETE /identity/v3/projects/3c81303b75d642c69cf80e9ecfb88724 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:06.903432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:06.905616 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:06.983966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:07.015048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:07.029996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 864/1705] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Jul 25 08:46:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:07.095051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fffec-368b-43b1-885b-a342f50a3915 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:07.101655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:07.102891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:07.103437 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:07.104838 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] PATH_INFO: `/v3/users/45493ff555104c65b3d9cac8679b173e` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:07.106828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=45493ff555104c65b3d9cac8679b173e)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:07.109029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:07.220473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45493ff555104c65b3d9cac8679b173e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:07.247940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45493ff555104c65b3d9cac8679b173e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:07.301100 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45493ff555104c65b3d9cac8679b173e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:07.338375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45493ff555104c65b3d9cac8679b173e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:07.339560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45493ff555104c65b3d9cac8679b173e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:07.370954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 842/1706] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:06 2022] DELETE /identity/v3/users/45493ff555104c65b3d9cac8679b173e => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:07.470632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a370efc-f723-4904-b5cb-fe4fcaee34d3 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:07.472706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7a370efc-f723-4904-b5cb-fe4fcaee34d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:07.507386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a370efc-f723-4904-b5cb-fe4fcaee34d3 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:07.710247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bf63b-9401-4fd6-b06e-7e34977563c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:07.711268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bf63b-9401-4fd6-b06e-7e34977563c0 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:07.711766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bf63b-9401-4fd6-b06e-7e34977563c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:07.712331 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bf63b-9401-4fd6-b06e-7e34977563c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:07.715123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-268bf63b-9401-4fd6-b06e-7e34977563c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:07.738770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bf63b-9401-4fd6-b06e-7e34977563c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:07.741579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bf63b-9401-4fd6-b06e-7e34977563c0 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:07.758653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 865/1707] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:08.525387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:08.527080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:08.770065 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e8bee-a718-4d45-bc6b-d0bfebe28c6e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:08.797578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:08.798742 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:08.799035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:08.799454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e admin admin] PATH_INFO: `/v3/projects/3f38a23055a84d6fb72057044c938075` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:08.809565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f38a23055a84d6fb72057044c938075)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:08.811873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:09.026024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f38a23055a84d6fb72057044c938075', 'request_id': 'req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:09.030919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:09.041213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f38a23055a84d6fb72057044c938075', 'request_id': 'req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:09.060081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f38a23055a84d6fb72057044c938075', 'request_id': 'req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:09.087854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e admin admin] The token cache is being invalidate because project 3f38a23055a84d6fb72057044c938075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:09.088386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:09.090001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2099f0ae-fcc1-4a7f-adbf-76918e973a6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:09.092908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 843/1708] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:08 2022] DELETE /identity/v3/projects/3f38a23055a84d6fb72057044c938075 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:09.098058 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5c68c-5a62-4f9f-b150-01bf7388513c None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:09.099112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5c68c-5a62-4f9f-b150-01bf7388513c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:09.099548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5c68c-5a62-4f9f-b150-01bf7388513c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:09.171343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a1c5c68c-5a62-4f9f-b150-01bf7388513c None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:09.180526 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a1c5c68c-5a62-4f9f-b150-01bf7388513c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:09.475819 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 866/1709] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:09.484980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b630c-2183-4c32-b3d3-cf19efe9b956 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:09.485930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b630c-2183-4c32-b3d3-cf19efe9b956 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:09.486744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b630c-2183-4c32-b3d3-cf19efe9b956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:09.534777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-8d3b630c-2183-4c32-b3d3-cf19efe9b956 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:09.537749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8d3b630c-2183-4c32-b3d3-cf19efe9b956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:09.567367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 844/1710] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:09.568427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5c68c-5a62-4f9f-b150-01bf7388513c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:09.585708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5c68c-5a62-4f9f-b150-01bf7388513c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:09.605329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e8a94-b20e-4cd9-8c66-587df47662ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:09.605987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e8a94-b20e-4cd9-8c66-587df47662ad admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:09.606350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e8a94-b20e-4cd9-8c66-587df47662ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:09.606721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e8a94-b20e-4cd9-8c66-587df47662ad admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:09.607917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e8a94-b20e-4cd9-8c66-587df47662ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:09.608237 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e8a94-b20e-4cd9-8c66-587df47662ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:09.610397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e8a94-b20e-4cd9-8c66-587df47662ad admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:09.654586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 867/1711] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:46:09.659377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b630c-2183-4c32-b3d3-cf19efe9b956 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:09.669274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b630c-2183-4c32-b3d3-cf19efe9b956 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:09.684924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bc6d0-1bd2-4312-a4e9-4aba3988ba78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:09.685622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bc6d0-1bd2-4312-a4e9-4aba3988ba78 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:09.685911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bc6d0-1bd2-4312-a4e9-4aba3988ba78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:09.686195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bc6d0-1bd2-4312-a4e9-4aba3988ba78 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:09.687455 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bc6d0-1bd2-4312-a4e9-4aba3988ba78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:09.690086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bc6d0-1bd2-4312-a4e9-4aba3988ba78 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:09.768671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 845/1712] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:09 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:09.772932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e8a94-b20e-4cd9-8c66-587df47662ad admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:09.799549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e8a94-b20e-4cd9-8c66-587df47662ad admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:09.927716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb048c8f-a5cb-43f5-a124-485cb8e87715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:09.929171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb048c8f-a5cb-43f5-a124-485cb8e87715 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:09.929613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb048c8f-a5cb-43f5-a124-485cb8e87715 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:09.930443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb048c8f-a5cb-43f5-a124-485cb8e87715 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:09.932109 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb048c8f-a5cb-43f5-a124-485cb8e87715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:09.935744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb048c8f-a5cb-43f5-a124-485cb8e87715 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:09.995129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 868/1713] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:09 2022] POST /identity/v3/users => generated 284 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:09.999816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bc6d0-1bd2-4312-a4e9-4aba3988ba78 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:10.035986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bc6d0-1bd2-4312-a4e9-4aba3988ba78 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:10.054894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f3b7-c2f0-43cf-8e5a-66f23bc45714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:10.055569 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f3b7-c2f0-43cf-8e5a-66f23bc45714 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:10.055864 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f3b7-c2f0-43cf-8e5a-66f23bc45714 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:10.056201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f3b7-c2f0-43cf-8e5a-66f23bc45714 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:10.056941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3f3b7-c2f0-43cf-8e5a-66f23bc45714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:10.085528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3f3b7-c2f0-43cf-8e5a-66f23bc45714 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:10.123065 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb048c8f-a5cb-43f5-a124-485cb8e87715 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:10.132017 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 846/1714] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:10.150790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb048c8f-a5cb-43f5-a124-485cb8e87715 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:10.165565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34642258-9d56-4b7d-8343-9b08bdc8fa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:10.167445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34642258-9d56-4b7d-8343-9b08bdc8fa34 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:10.168923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34642258-9d56-4b7d-8343-9b08bdc8fa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:10.169348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34642258-9d56-4b7d-8343-9b08bdc8fa34 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:10.170274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34642258-9d56-4b7d-8343-9b08bdc8fa34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:10.173126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34642258-9d56-4b7d-8343-9b08bdc8fa34 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:10.209637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 869/1715] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:10.213416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f3b7-c2f0-43cf-8e5a-66f23bc45714 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:10.243152 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f3b7-c2f0-43cf-8e5a-66f23bc45714 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:10.253743 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95917d7f-b02b-449d-9678-a215533af353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:10.254752 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95917d7f-b02b-449d-9678-a215533af353 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:10.255047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95917d7f-b02b-449d-9678-a215533af353 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:10.255458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95917d7f-b02b-449d-9678-a215533af353 admin admin] PATH_INFO: `/v3/projects/c1790fa30d5d4d9485699dd95e188337/users/9b5d000e3f5545e69944cf9e87cd8dad/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:10.293719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34642258-9d56-4b7d-8343-9b08bdc8fa34 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:10.296875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-34642258-9d56-4b7d-8343-9b08bdc8fa34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:10.303808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95917d7f-b02b-449d-9678-a215533af353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1790fa30d5d4d9485699dd95e188337, user_id=9b5d000e3f5545e69944cf9e87cd8dad, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:10.310008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95917d7f-b02b-449d-9678-a215533af353 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:10.318445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34642258-9d56-4b7d-8343-9b08bdc8fa34 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:10.357789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 847/1716] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:10 2022] PUT /identity/v3/projects/c1790fa30d5d4d9485699dd95e188337/users/9b5d000e3f5545e69944cf9e87cd8dad/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:10.359390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bd748-25ad-4097-972d-48ef2da0f6b6 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:10.359688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bd748-25ad-4097-972d-48ef2da0f6b6 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:10.359974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bd748-25ad-4097-972d-48ef2da0f6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:10.362147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18d354-2eca-4fc3-9f54-3f1992c69ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:10.363952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18d354-2eca-4fc3-9f54-3f1992c69ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:10.364532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18d354-2eca-4fc3-9f54-3f1992c69ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:10.365185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18d354-2eca-4fc3-9f54-3f1992c69ac5 admin admin] PATH_INFO: `/v3/users/95487356edb841b387df2a31b519a740` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:10.367180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18d354-2eca-4fc3-9f54-3f1992c69ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95487356edb841b387df2a31b519a740)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:10.369959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18d354-2eca-4fc3-9f54-3f1992c69ac5 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:10.372650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:10.452055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-253bd748-25ad-4097-972d-48ef2da0f6b6 None None] MFA Rules not processed for user `9b5d000e3f5545e69944cf9e87cd8dad`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:10.456006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-253bd748-25ad-4097-972d-48ef2da0f6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:10.521187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-de18d354-2eca-4fc3-9f54-3f1992c69ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95487356edb841b387df2a31b519a740'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:10.529099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-de18d354-2eca-4fc3-9f54-3f1992c69ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95487356edb841b387df2a31b519a740'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:10.545367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-de18d354-2eca-4fc3-9f54-3f1992c69ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95487356edb841b387df2a31b519a740'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:10.591976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-de18d354-2eca-4fc3-9f54-3f1992c69ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95487356edb841b387df2a31b519a740'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:10.638869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-de18d354-2eca-4fc3-9f54-3f1992c69ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95487356edb841b387df2a31b519a740'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:10.669981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 870/1717] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:10 2022] DELETE /identity/v3/users/95487356edb841b387df2a31b519a740 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:10.675552 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf1b03-3346-42b7-a88c-25fd80e4602d None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:10.676001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf1b03-3346-42b7-a88c-25fd80e4602d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:10.677102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf1b03-3346-42b7-a88c-25fd80e4602d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:10.693297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 848/1718] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:10.693297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bd748-25ad-4097-972d-48ef2da0f6b6 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:10.773383 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d5cf1b03-3346-42b7-a88c-25fd80e4602d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:10.779424 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d5cf1b03-3346-42b7-a88c-25fd80e4602d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:10.864928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bd748-25ad-4097-972d-48ef2da0f6b6 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:10.882841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:11.055428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948080d-81b7-4328-b2d9-f102fbc0cd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:11.056356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948080d-81b7-4328-b2d9-f102fbc0cd91 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:11.057009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948080d-81b7-4328-b2d9-f102fbc0cd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:11.057395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948080d-81b7-4328-b2d9-f102fbc0cd91 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:11.058349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948080d-81b7-4328-b2d9-f102fbc0cd91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:11.061727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948080d-81b7-4328-b2d9-f102fbc0cd91 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:11.082763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 871/1719] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:11.097947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf1b03-3346-42b7-a88c-25fd80e4602d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:11.100909 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d5cf1b03-3346-42b7-a88c-25fd80e4602d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:11.154264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 849/1720] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:10 2022] POST /identity/v3/projects => generated 367 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:11.158896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948080d-81b7-4328-b2d9-f102fbc0cd91 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:11.167960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-9948080d-81b7-4328-b2d9-f102fbc0cd91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:11.257911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf1b03-3346-42b7-a88c-25fd80e4602d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:11.309663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948080d-81b7-4328-b2d9-f102fbc0cd91 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:11.310488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:11.319584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba59e6-3d7a-4223-a98f-9f0ae9c04851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:11.320516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba59e6-3d7a-4223-a98f-9f0ae9c04851 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:11.320993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba59e6-3d7a-4223-a98f-9f0ae9c04851 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:11.321422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba59e6-3d7a-4223-a98f-9f0ae9c04851 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:11.323533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba59e6-3d7a-4223-a98f-9f0ae9c04851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:11.325873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba59e6-3d7a-4223-a98f-9f0ae9c04851 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:11.337114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceac315-dac3-44b8-8114-0d58cb794b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:11.337864 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceac315-dac3-44b8-8114-0d58cb794b12 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:11.338160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceac315-dac3-44b8-8114-0d58cb794b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:11.338445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceac315-dac3-44b8-8114-0d58cb794b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:11.349275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceac315-dac3-44b8-8114-0d58cb794b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:11.350843 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceac315-dac3-44b8-8114-0d58cb794b12 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:11.368663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 872/1721] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:11.378114 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 850/1722] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:11 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:11.383977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceac315-dac3-44b8-8114-0d58cb794b12 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:11.404160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceac315-dac3-44b8-8114-0d58cb794b12 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:11.411030 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed912b9-d0b3-4fb9-99fa-0032258474d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:11.411742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed912b9-d0b3-4fb9-99fa-0032258474d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:11.412031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed912b9-d0b3-4fb9-99fa-0032258474d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:11.412358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed912b9-d0b3-4fb9-99fa-0032258474d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:11.413165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed912b9-d0b3-4fb9-99fa-0032258474d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:11.414903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed912b9-d0b3-4fb9-99fa-0032258474d2 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:11.437871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba59e6-3d7a-4223-a98f-9f0ae9c04851 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:11.443814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 873/1723] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:11.449420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba59e6-3d7a-4223-a98f-9f0ae9c04851 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:11.465408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a13c4-feee-449d-baad-2dc6436bbf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:11.467211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a13c4-feee-449d-baad-2dc6436bbf63 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:11.467950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a13c4-feee-449d-baad-2dc6436bbf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:11.473666 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a13c4-feee-449d-baad-2dc6436bbf63 admin admin] PATH_INFO: `/v3/projects/4c4dcf774e514b88b27042534ad02fd1/users/c79ccca2efdf42369224bf0e9b80aed2/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:11.513734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a13c4-feee-449d-baad-2dc6436bbf63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c4dcf774e514b88b27042534ad02fd1, user_id=c79ccca2efdf42369224bf0e9b80aed2, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:11.518107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a13c4-feee-449d-baad-2dc6436bbf63 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:11.582696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 851/1724] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:11 2022] PUT /identity/v3/projects/4c4dcf774e514b88b27042534ad02fd1/users/c79ccca2efdf42369224bf0e9b80aed2/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:11.589600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed912b9-d0b3-4fb9-99fa-0032258474d2 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:11.669286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed912b9-d0b3-4fb9-99fa-0032258474d2 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:11.945384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcd583-56b5-47cc-963d-bd3b68f9328d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:11.946303 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcd583-56b5-47cc-963d-bd3b68f9328d admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:11.946697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcd583-56b5-47cc-963d-bd3b68f9328d admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:11.947099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcd583-56b5-47cc-963d-bd3b68f9328d admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:11.949093 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcd583-56b5-47cc-963d-bd3b68f9328d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:11.956972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcd583-56b5-47cc-963d-bd3b68f9328d admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:12.023332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 874/1725] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:12.027781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a13c4-feee-449d-baad-2dc6436bbf63 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:12.053053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a13c4-feee-449d-baad-2dc6436bbf63 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:12.069307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2a9f1-420f-4fc5-b0c0-255b9bfdf0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:12.072363 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2a9f1-420f-4fc5-b0c0-255b9bfdf0a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:12.074225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2a9f1-420f-4fc5-b0c0-255b9bfdf0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:12.074748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2a9f1-420f-4fc5-b0c0-255b9bfdf0a4 admin admin] PATH_INFO: `/v3/projects/4c4dcf774e514b88b27042534ad02fd1/users/c79ccca2efdf42369224bf0e9b80aed2/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:12.082581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2a9f1-420f-4fc5-b0c0-255b9bfdf0a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c4dcf774e514b88b27042534ad02fd1, user_id=c79ccca2efdf42369224bf0e9b80aed2, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:12.090582 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2a9f1-420f-4fc5-b0c0-255b9bfdf0a4 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:12.148239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 852/1726] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:12 2022] PUT /identity/v3/projects/4c4dcf774e514b88b27042534ad02fd1/users/c79ccca2efdf42369224bf0e9b80aed2/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:12.150393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63418035-200e-4281-94c3-d6813c85304f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:12.150690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63418035-200e-4281-94c3-d6813c85304f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:12.151416 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63418035-200e-4281-94c3-d6813c85304f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:12.266708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-63418035-200e-4281-94c3-d6813c85304f None None] MFA Rules not processed for user `c79ccca2efdf42369224bf0e9b80aed2`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:12.267629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-63418035-200e-4281-94c3-d6813c85304f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:12.488817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2a9f1-420f-4fc5-b0c0-255b9bfdf0a4 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:12.490886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-63f2a9f1-420f-4fc5-b0c0-255b9bfdf0a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:12.501391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 875/1727] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:12.517339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63418035-200e-4281-94c3-d6813c85304f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:12.519784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-63418035-200e-4281-94c3-d6813c85304f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:12.608362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:12.643680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:12.649648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2a9f1-420f-4fc5-b0c0-255b9bfdf0a4 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:12.653477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63418035-200e-4281-94c3-d6813c85304f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:12.661837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5413fd7-89ac-472b-bec4-271618cc43e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:12.663125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c6620-a203-4ded-be32-0ead11ffd53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:12.663943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5413fd7-89ac-472b-bec4-271618cc43e3 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:12.665013 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5413fd7-89ac-472b-bec4-271618cc43e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:12.665670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c6620-a203-4ded-be32-0ead11ffd53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:12.666113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5413fd7-89ac-472b-bec4-271618cc43e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:12.667246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c6620-a203-4ded-be32-0ead11ffd53f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:12.667684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c6620-a203-4ded-be32-0ead11ffd53f admin admin] PATH_INFO: `/v3/users/6dbc3275520844edb074078eb8f54c4b` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:12.670759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c6620-a203-4ded-be32-0ead11ffd53f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dbc3275520844edb074078eb8f54c4b)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:12.678502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c6620-a203-4ded-be32-0ead11ffd53f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:12.692110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5413fd7-89ac-472b-bec4-271618cc43e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:12.693849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5413fd7-89ac-472b-bec4-271618cc43e3 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:12.734785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 876/1728] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:12.801306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5413fd7-89ac-472b-bec4-271618cc43e3 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:12.803182 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e5413fd7-89ac-472b-bec4-271618cc43e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:12.827271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5413fd7-89ac-472b-bec4-271618cc43e3 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:12.830904 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2b0c6620-a203-4ded-be32-0ead11ffd53f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dbc3275520844edb074078eb8f54c4b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:12.841714 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a948d-3607-4786-8302-572d526007f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:12.844890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a948d-3607-4786-8302-572d526007f6 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:12.845585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a948d-3607-4786-8302-572d526007f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:12.845997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a948d-3607-4786-8302-572d526007f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:12.850626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ba6a948d-3607-4786-8302-572d526007f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:12.852884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2b0c6620-a203-4ded-be32-0ead11ffd53f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dbc3275520844edb074078eb8f54c4b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:12.906159 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2b0c6620-a203-4ded-be32-0ead11ffd53f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dbc3275520844edb074078eb8f54c4b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:12.938990 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2b0c6620-a203-4ded-be32-0ead11ffd53f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dbc3275520844edb074078eb8f54c4b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:12.941500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2b0c6620-a203-4ded-be32-0ead11ffd53f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dbc3275520844edb074078eb8f54c4b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:12.979812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a948d-3607-4786-8302-572d526007f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:12.981515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a948d-3607-4786-8302-572d526007f6 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:12.982831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 853/1729] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:12 2022] DELETE /identity/v3/users/6dbc3275520844edb074078eb8f54c4b => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:12.990953 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f0c0-1edc-4bfe-af9b-4e4036ddd176 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:12.993803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f0c0-1edc-4bfe-af9b-4e4036ddd176 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:12.995216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f0c0-1edc-4bfe-af9b-4e4036ddd176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:13.013078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 877/1730] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:46:13.097028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-09e1f0c0-1edc-4bfe-af9b-4e4036ddd176 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:13.102700 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-09e1f0c0-1edc-4bfe-af9b-4e4036ddd176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:13.332866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95435980-0a09-4810-8d72-cc48a5c37132 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:13.334185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95435980-0a09-4810-8d72-cc48a5c37132 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:13.335116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95435980-0a09-4810-8d72-cc48a5c37132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:13.371192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-95435980-0a09-4810-8d72-cc48a5c37132 None None] MFA Rules not processed for user `8e682232b7374450a7265e092602a5a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:13.376281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-95435980-0a09-4810-8d72-cc48a5c37132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:13.396795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 854/1731] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:13.419076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f0c0-1edc-4bfe-af9b-4e4036ddd176 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:13.420580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-09e1f0c0-1edc-4bfe-af9b-4e4036ddd176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:13.646069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 878/1732] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:13.647379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95435980-0a09-4810-8d72-cc48a5c37132 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:13.650677 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-95435980-0a09-4810-8d72-cc48a5c37132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:13.654328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f0c0-1edc-4bfe-af9b-4e4036ddd176 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:13.684219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9a3d7-1e0a-4d9f-bace-5a7b8e14821b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:13.686626 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9a3d7-1e0a-4d9f-bace-5a7b8e14821b service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:13.688334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9a3d7-1e0a-4d9f-bace-5a7b8e14821b service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:13.704921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9a3d7-1e0a-4d9f-bace-5a7b8e14821b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:13.717446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95435980-0a09-4810-8d72-cc48a5c37132 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:13.734392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24f08f-438c-421d-8096-b6f4f6118e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:13.735906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24f08f-438c-421d-8096-b6f4f6118e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:13.736202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24f08f-438c-421d-8096-b6f4f6118e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:13.737846 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24f08f-438c-421d-8096-b6f4f6118e6b admin admin] PATH_INFO: `/v3/projects/b04bad9c05ec40a4a5975e9e03a1f4d2` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:13.740349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24f08f-438c-421d-8096-b6f4f6118e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04bad9c05ec40a4a5975e9e03a1f4d2)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:13.747325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24f08f-438c-421d-8096-b6f4f6118e6b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:13.779008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9a3d7-1e0a-4d9f-bace-5a7b8e14821b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:13.789082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9a3d7-1e0a-4d9f-bace-5a7b8e14821b service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:13.849058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 855/1733] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:13.855283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ce547-8bfa-484f-9622-a9797b9d3d15 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:13.858226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ce547-8bfa-484f-9622-a9797b9d3d15 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:13.860376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ce547-8bfa-484f-9622-a9797b9d3d15 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:13.864021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 856/1734] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:46:13 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:46:13.886439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-dd24f08f-438c-421d-8096-b6f4f6118e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04bad9c05ec40a4a5975e9e03a1f4d2', 'request_id': 'req-dd24f08f-438c-421d-8096-b6f4f6118e6b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:13.904414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-dd24f08f-438c-421d-8096-b6f4f6118e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04bad9c05ec40a4a5975e9e03a1f4d2', 'request_id': 'req-dd24f08f-438c-421d-8096-b6f4f6118e6b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:13.914690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-dd24f08f-438c-421d-8096-b6f4f6118e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04bad9c05ec40a4a5975e9e03a1f4d2', 'request_id': 'req-dd24f08f-438c-421d-8096-b6f4f6118e6b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:13.941318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-dd24f08f-438c-421d-8096-b6f4f6118e6b admin admin] The token cache is being invalidate because project b04bad9c05ec40a4a5975e9e03a1f4d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:13.941601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-dd24f08f-438c-421d-8096-b6f4f6118e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:13.953040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-dd24f08f-438c-421d-8096-b6f4f6118e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:13.955903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 879/1735] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:13 2022] DELETE /identity/v3/projects/b04bad9c05ec40a4a5975e9e03a1f4d2 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:13.957670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f294f25-d8d3-4737-a50f-53a2ed81e2ef None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:13.957980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f294f25-d8d3-4737-a50f-53a2ed81e2ef None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:13.958333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f294f25-d8d3-4737-a50f-53a2ed81e2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:13.996295 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-9f294f25-d8d3-4737-a50f-53a2ed81e2ef None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:13.999939 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-9f294f25-d8d3-4737-a50f-53a2ed81e2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:14.010458 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03a713-18b5-4bf3-ae79-4644e0a0f92c None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:14.011000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03a713-18b5-4bf3-ae79-4644e0a0f92c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:14.011294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03a713-18b5-4bf3-ae79-4644e0a0f92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:14.016999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:14.069769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-0d03a713-18b5-4bf3-ae79-4644e0a0f92c None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:14.071045 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0d03a713-18b5-4bf3-ae79-4644e0a0f92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:14.093268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:14.194688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 857/1736] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:14.202379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f709d33-cc2c-451d-9294-5c764bcb2346 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:14.203246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f709d33-cc2c-451d-9294-5c764bcb2346 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:14.203701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f709d33-cc2c-451d-9294-5c764bcb2346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:14.294442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 880/1737] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:46:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:14.318865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-6f709d33-cc2c-451d-9294-5c764bcb2346 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:14.321708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-6f709d33-cc2c-451d-9294-5c764bcb2346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:14.361388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 858/1738] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:14.363404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03a713-18b5-4bf3-ae79-4644e0a0f92c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:14.380233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03a713-18b5-4bf3-ae79-4644e0a0f92c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:14.399348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90f345-6851-4c73-9d3e-72655454ddf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:14.401281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90f345-6851-4c73-9d3e-72655454ddf8 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:14.403189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90f345-6851-4c73-9d3e-72655454ddf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:14.403686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90f345-6851-4c73-9d3e-72655454ddf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:14.407632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90f345-6851-4c73-9d3e-72655454ddf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:14.410109 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90f345-6851-4c73-9d3e-72655454ddf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:14.414525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90f345-6851-4c73-9d3e-72655454ddf8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:14.449887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 881/1739] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:46:14.453481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f709d33-cc2c-451d-9294-5c764bcb2346 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:14.466025 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f709d33-cc2c-451d-9294-5c764bcb2346 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:14.482374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2d6b9-7446-44be-8e5f-d9e5486dce8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:14.483035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2d6b9-7446-44be-8e5f-d9e5486dce8e admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:14.483325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2d6b9-7446-44be-8e5f-d9e5486dce8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:14.483668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2d6b9-7446-44be-8e5f-d9e5486dce8e admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:14.484921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2d6b9-7446-44be-8e5f-d9e5486dce8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:14.486865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2d6b9-7446-44be-8e5f-d9e5486dce8e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:14.551048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90f345-6851-4c73-9d3e-72655454ddf8 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:14.551554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 859/1740] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:14 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:14.586951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90f345-6851-4c73-9d3e-72655454ddf8 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:14.791696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7ae3f-4501-411a-95e5-01535e0311a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:14.793371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7ae3f-4501-411a-95e5-01535e0311a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:14.793909 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7ae3f-4501-411a-95e5-01535e0311a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:14.794515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7ae3f-4501-411a-95e5-01535e0311a4 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:14.796269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7ae3f-4501-411a-95e5-01535e0311a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:14.799495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7ae3f-4501-411a-95e5-01535e0311a4 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:14.836625 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 882/1741] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:14 2022] POST /identity/v3/users => generated 282 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:14.838951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2d6b9-7446-44be-8e5f-d9e5486dce8e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:14.860242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2d6b9-7446-44be-8e5f-d9e5486dce8e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:14.872899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10516f-a74b-4de7-8a56-f03ac009f2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:14.874376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10516f-a74b-4de7-8a56-f03ac009f2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:14.879100 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10516f-a74b-4de7-8a56-f03ac009f2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:14.889021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10516f-a74b-4de7-8a56-f03ac009f2c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:14.895727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10516f-a74b-4de7-8a56-f03ac009f2c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:14.897880 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10516f-a74b-4de7-8a56-f03ac009f2c9 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:14.928755 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 860/1742] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:14.929651 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7ae3f-4501-411a-95e5-01535e0311a4 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:14.946774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7ae3f-4501-411a-95e5-01535e0311a4 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:14.961903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519fddc-4539-43d9-b25b-771878375dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:14.962634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519fddc-4539-43d9-b25b-771878375dcc admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:14.962866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519fddc-4539-43d9-b25b-771878375dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:14.963148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519fddc-4539-43d9-b25b-771878375dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:14.964255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519fddc-4539-43d9-b25b-771878375dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:14.968884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519fddc-4539-43d9-b25b-771878375dcc admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:14.985068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 883/1743] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:14.989997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10516f-a74b-4de7-8a56-f03ac009f2c9 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:15.002273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b863d3e-6fe1-40f7-b0fd-6e0cc70b0c65 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:15.002522 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b863d3e-6fe1-40f7-b0fd-6e0cc70b0c65 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:15.003553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10516f-a74b-4de7-8a56-f03ac009f2c9 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:15.004422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b863d3e-6fe1-40f7-b0fd-6e0cc70b0c65 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:15.006979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 884/1744] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:46:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:46:15.011178 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75beb56c-1333-4eeb-ab13-3532fbe156f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:15.012099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beb56c-1333-4eeb-ab13-3532fbe156f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:15.012385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beb56c-1333-4eeb-ab13-3532fbe156f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:15.012763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beb56c-1333-4eeb-ab13-3532fbe156f9 admin admin] PATH_INFO: `/v3/projects/908a6c6784b14cffb4498e48f0c393be/users/9e8677b9ea3a438eba67406eda45f878/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:15.050429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75beb56c-1333-4eeb-ab13-3532fbe156f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=908a6c6784b14cffb4498e48f0c393be, user_id=9e8677b9ea3a438eba67406eda45f878, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:15.053256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75beb56c-1333-4eeb-ab13-3532fbe156f9 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:15.094495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 861/1745] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:14 2022] PUT /identity/v3/projects/908a6c6784b14cffb4498e48f0c393be/users/9e8677b9ea3a438eba67406eda45f878/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:15.099346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c8988-ae9a-42f9-afeb-bbdbdd89fae7 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:15.100265 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c8988-ae9a-42f9-afeb-bbdbdd89fae7 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:15.100937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c8988-ae9a-42f9-afeb-bbdbdd89fae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:15.183630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-fc9c8988-ae9a-42f9-afeb-bbdbdd89fae7 None None] MFA Rules not processed for user `9e8677b9ea3a438eba67406eda45f878`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:15.185948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fc9c8988-ae9a-42f9-afeb-bbdbdd89fae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:15.246137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a8aa7-2690-4882-ab80-719a2bf4dcdc None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:15.246137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a8aa7-2690-4882-ab80-719a2bf4dcdc None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:15.246137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a8aa7-2690-4882-ab80-719a2bf4dcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:15.334958 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-389a8aa7-2690-4882-ab80-719a2bf4dcdc None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:15.337106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-389a8aa7-2690-4882-ab80-719a2bf4dcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:15.497799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 885/1746] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:15.498757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c8988-ae9a-42f9-afeb-bbdbdd89fae7 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:15.522543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:15.629981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 862/1747] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:46:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:15.742196 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c8988-ae9a-42f9-afeb-bbdbdd89fae7 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:15.752986 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601cd7b-69b2-4ea7-90d5-d66c6c4b574e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:15.755283 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601cd7b-69b2-4ea7-90d5-d66c6c4b574e admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:15.756688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601cd7b-69b2-4ea7-90d5-d66c6c4b574e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:15.757187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601cd7b-69b2-4ea7-90d5-d66c6c4b574e admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:15.764666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601cd7b-69b2-4ea7-90d5-d66c6c4b574e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:15.771050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601cd7b-69b2-4ea7-90d5-d66c6c4b574e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:15.885970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 886/1748] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:15 2022] POST /identity/v3/projects => generated 365 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:15.888652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a8aa7-2690-4882-ab80-719a2bf4dcdc None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:15.911382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a8aa7-2690-4882-ab80-719a2bf4dcdc None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:16.108944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:16.165109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb90dc2-a8d0-42f1-8aa2-7502c5cca3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:16.173635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb90dc2-a8d0-42f1-8aa2-7502c5cca3b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:16.190486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb90dc2-a8d0-42f1-8aa2-7502c5cca3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:16.190486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb90dc2-a8d0-42f1-8aa2-7502c5cca3b2 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:16.190486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb90dc2-a8d0-42f1-8aa2-7502c5cca3b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:16.190486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb90dc2-a8d0-42f1-8aa2-7502c5cca3b2 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:16.240943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 863/1749] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:15 2022] POST /identity/v3/users => generated 289 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:16.245075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601cd7b-69b2-4ea7-90d5-d66c6c4b574e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:16.259658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601cd7b-69b2-4ea7-90d5-d66c6c4b574e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:16.278163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6009-9e43-45a0-bbf9-b4b54b93f0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:16.279095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6009-9e43-45a0-bbf9-b4b54b93f0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:16.279598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6009-9e43-45a0-bbf9-b4b54b93f0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:16.280051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6009-9e43-45a0-bbf9-b4b54b93f0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:16.281890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6009-9e43-45a0-bbf9-b4b54b93f0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:16.284140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6009-9e43-45a0-bbf9-b4b54b93f0c3 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:16.297972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 887/1750] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:16.301935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb90dc2-a8d0-42f1-8aa2-7502c5cca3b2 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:16.325587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb90dc2-a8d0-42f1-8aa2-7502c5cca3b2 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:16.332445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9536204-3b93-49ac-9aa5-db400455ba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:16.335855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9536204-3b93-49ac-9aa5-db400455ba59 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:16.336475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9536204-3b93-49ac-9aa5-db400455ba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:16.337302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9536204-3b93-49ac-9aa5-db400455ba59 admin admin] PATH_INFO: `/v3/projects/89beadda94d145338daf12626737c568/users/d5c31ef3f425476087f5b6e7181e0352/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:16.385276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9536204-3b93-49ac-9aa5-db400455ba59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89beadda94d145338daf12626737c568, user_id=d5c31ef3f425476087f5b6e7181e0352, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:16.394593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9536204-3b93-49ac-9aa5-db400455ba59 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:16.429191 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 864/1751] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:16 2022] PUT /identity/v3/projects/89beadda94d145338daf12626737c568/users/d5c31ef3f425476087f5b6e7181e0352/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:16.433341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6009-9e43-45a0-bbf9-b4b54b93f0c3 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:16.447894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6009-9e43-45a0-bbf9-b4b54b93f0c3 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:16.509837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9536204-3b93-49ac-9aa5-db400455ba59 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:16.512772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c9536204-3b93-49ac-9aa5-db400455ba59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:16.552906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8705532-7ef6-43e2-9610-da411fe5b01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:16.554554 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8705532-7ef6-43e2-9610-da411fe5b01b admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:16.554686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8705532-7ef6-43e2-9610-da411fe5b01b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:16.555121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8705532-7ef6-43e2-9610-da411fe5b01b admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:16.555987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8705532-7ef6-43e2-9610-da411fe5b01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:16.557940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8705532-7ef6-43e2-9610-da411fe5b01b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:16.570859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 888/1752] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:16.576402 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8705532-7ef6-43e2-9610-da411fe5b01b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:16.607119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8705532-7ef6-43e2-9610-da411fe5b01b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:16.617473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e0296-3bed-40a8-be45-782aac80f0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:16.618543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e0296-3bed-40a8-be45-782aac80f0f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:16.618954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e0296-3bed-40a8-be45-782aac80f0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:16.619351 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e0296-3bed-40a8-be45-782aac80f0f0 admin admin] PATH_INFO: `/v3/projects/89beadda94d145338daf12626737c568/users/d5c31ef3f425476087f5b6e7181e0352/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:16.621800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e0296-3bed-40a8-be45-782aac80f0f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89beadda94d145338daf12626737c568, user_id=d5c31ef3f425476087f5b6e7181e0352, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:16.624822 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e0296-3bed-40a8-be45-782aac80f0f0 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:16.643868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9536204-3b93-49ac-9aa5-db400455ba59 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:16.648211 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 889/1753] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:16 2022] PUT /identity/v3/projects/89beadda94d145338daf12626737c568/users/d5c31ef3f425476087f5b6e7181e0352/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:16.659572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544c93c-aaf5-4e2b-8665-89ab7b1901cb None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:16.660009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544c93c-aaf5-4e2b-8665-89ab7b1901cb None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:16.660955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544c93c-aaf5-4e2b-8665-89ab7b1901cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:16.713726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-3544c93c-aaf5-4e2b-8665-89ab7b1901cb None None] MFA Rules not processed for user `d5c31ef3f425476087f5b6e7181e0352`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:16.714891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3544c93c-aaf5-4e2b-8665-89ab7b1901cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:16.719590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:16.720550 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:16.720969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:16.721502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] PATH_INFO: `/v3/projects/6924bf00364e4ce9aef4da1854307ac2` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:16.724815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6924bf00364e4ce9aef4da1854307ac2)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:16.727349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:16.793941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:16.795777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6924bf00364e4ce9aef4da1854307ac2', 'request_id': 'req-50bc6f72-037d-4477-b109-cc2f695a5cc2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:16.803460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:16.809311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6924bf00364e4ce9aef4da1854307ac2', 'request_id': 'req-50bc6f72-037d-4477-b109-cc2f695a5cc2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:16.824971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 890/1754] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:16.825162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6924bf00364e4ce9aef4da1854307ac2', 'request_id': 'req-50bc6f72-037d-4477-b109-cc2f695a5cc2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:16.837565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544c93c-aaf5-4e2b-8665-89ab7b1901cb None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:16.838993 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3544c93c-aaf5-4e2b-8665-89ab7b1901cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:16.842077 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] The token cache is being invalidate because project 6924bf00364e4ce9aef4da1854307ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:16.842455 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:16.844239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:16.851704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 865/1755] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:16 2022] DELETE /identity/v3/projects/6924bf00364e4ce9aef4da1854307ac2 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:16.853156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:16.854622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:16.923495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544c93c-aaf5-4e2b-8665-89ab7b1901cb None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:16.946462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc6f72-037d-4477-b109-cc2f695a5cc2 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:16.957215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:16.958233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:16.958585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:16.958933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] PATH_INFO: `/v3/users/0ec61b626a2c445696cf70cab6f3c7dc` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:16.960358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec61b626a2c445696cf70cab6f3c7dc)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:16.962570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:16.988627 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bae983-1406-44fa-b234-9f1918a88579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:16.989315 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bae983-1406-44fa-b234-9f1918a88579 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:16.989727 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bae983-1406-44fa-b234-9f1918a88579 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:16.990032 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bae983-1406-44fa-b234-9f1918a88579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:16.992286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-96bae983-1406-44fa-b234-9f1918a88579 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:17.019379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec61b626a2c445696cf70cab6f3c7dc'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:17.029168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec61b626a2c445696cf70cab6f3c7dc'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:17.048721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec61b626a2c445696cf70cab6f3c7dc'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:17.060667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec61b626a2c445696cf70cab6f3c7dc'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:17.066528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec61b626a2c445696cf70cab6f3c7dc'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:17.081748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 866/1756] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:16 2022] DELETE /identity/v3/users/0ec61b626a2c445696cf70cab6f3c7dc => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:17.085000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:17.085000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:17.101773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bae983-1406-44fa-b234-9f1918a88579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:17.104377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bae983-1406-44fa-b234-9f1918a88579 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:17.119657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 891/1757] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:17.184431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:17.215887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de61ad8-e438-40b3-8b69-c8a73a675166 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:17.223236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60918e9d-ccbe-4430-acc7-44316d62ae8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:17.223681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60918e9d-ccbe-4430-acc7-44316d62ae8a service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:17.223954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60918e9d-ccbe-4430-acc7-44316d62ae8a service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:17.224791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60918e9d-ccbe-4430-acc7-44316d62ae8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:17.226002 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bae983-1406-44fa-b234-9f1918a88579 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:17.227418 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-96bae983-1406-44fa-b234-9f1918a88579 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:17.238100 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-60918e9d-ccbe-4430-acc7-44316d62ae8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:17.243877 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bae983-1406-44fa-b234-9f1918a88579 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:17.259335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60918e9d-ccbe-4430-acc7-44316d62ae8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:17.261910 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60918e9d-ccbe-4430-acc7-44316d62ae8a service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:17.278634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 867/1758] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:17.340757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e4190-77aa-45d9-be14-5886092424dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:17.341771 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e4190-77aa-45d9-be14-5886092424dd service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:17.342071 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e4190-77aa-45d9-be14-5886092424dd service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:17.343653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e4190-77aa-45d9-be14-5886092424dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:17.345872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-198e4190-77aa-45d9-be14-5886092424dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:17.354489 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:17.492001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60918e9d-ccbe-4430-acc7-44316d62ae8a service nova] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:17.497024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-60918e9d-ccbe-4430-acc7-44316d62ae8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:17.512338 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e4190-77aa-45d9-be14-5886092424dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:17.513895 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e4190-77aa-45d9-be14-5886092424dd service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:17.553541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 892/1759] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:17 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:17.607147 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60918e9d-ccbe-4430-acc7-44316d62ae8a service nova] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:17.612464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc99a5-1098-4307-952e-c271b84d9b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:17.613396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc99a5-1098-4307-952e-c271b84d9b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:17.613682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc99a5-1098-4307-952e-c271b84d9b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:17.613957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc99a5-1098-4307-952e-c271b84d9b1e admin admin] PATH_INFO: `/v3/projects/36d97558e37e422e87c96c6e21daec56` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:17.615299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc99a5-1098-4307-952e-c271b84d9b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d97558e37e422e87c96c6e21daec56)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:17.617483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc99a5-1098-4307-952e-c271b84d9b1e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:17.699718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-59fc99a5-1098-4307-952e-c271b84d9b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d97558e37e422e87c96c6e21daec56', 'request_id': 'req-59fc99a5-1098-4307-952e-c271b84d9b1e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:17.717052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-59fc99a5-1098-4307-952e-c271b84d9b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d97558e37e422e87c96c6e21daec56', 'request_id': 'req-59fc99a5-1098-4307-952e-c271b84d9b1e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:17.731458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-59fc99a5-1098-4307-952e-c271b84d9b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d97558e37e422e87c96c6e21daec56', 'request_id': 'req-59fc99a5-1098-4307-952e-c271b84d9b1e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:17.747786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-59fc99a5-1098-4307-952e-c271b84d9b1e admin admin] The token cache is being invalidate because project 36d97558e37e422e87c96c6e21daec56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:17.748165 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-59fc99a5-1098-4307-952e-c271b84d9b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:17.750730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-59fc99a5-1098-4307-952e-c271b84d9b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:17.753316 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 868/1760] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:17 2022] DELETE /identity/v3/projects/36d97558e37e422e87c96c6e21daec56 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:17.760476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec85ab-6656-418d-9ccb-c1de28ed30f0 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:17.761091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec85ab-6656-418d-9ccb-c1de28ed30f0 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:17.761427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec85ab-6656-418d-9ccb-c1de28ed30f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:17.814561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-03ec85ab-6656-418d-9ccb-c1de28ed30f0 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:17.817536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-03ec85ab-6656-418d-9ccb-c1de28ed30f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:17.905563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 893/1761] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:17.914238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d02974-4f03-48e8-943d-6a8939dcd049 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:17.917587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d02974-4f03-48e8-943d-6a8939dcd049 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:17.917587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d02974-4f03-48e8-943d-6a8939dcd049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:17.943244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-42d02974-4f03-48e8-943d-6a8939dcd049 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:17.944982 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-42d02974-4f03-48e8-943d-6a8939dcd049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:17.959331 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec85ab-6656-418d-9ccb-c1de28ed30f0 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:17.975747 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-03ec85ab-6656-418d-9ccb-c1de28ed30f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:17.982660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 869/1762] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:17.984252 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d02974-4f03-48e8-943d-6a8939dcd049 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:18.021204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d02974-4f03-48e8-943d-6a8939dcd049 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:18.031905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a8681-53af-40ad-8fb0-e32293bde570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:18.032688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a8681-53af-40ad-8fb0-e32293bde570 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:18.033066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a8681-53af-40ad-8fb0-e32293bde570 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:18.033384 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a8681-53af-40ad-8fb0-e32293bde570 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:18.034142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a8681-53af-40ad-8fb0-e32293bde570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:18.034452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a8681-53af-40ad-8fb0-e32293bde570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:18.043777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a8681-53af-40ad-8fb0-e32293bde570 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:18.071658 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 870/1763] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:46:18.078465 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a8681-53af-40ad-8fb0-e32293bde570 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:18.090844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a8681-53af-40ad-8fb0-e32293bde570 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:18.099311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448388e-a1fe-4ef8-ac0c-df3528bd97a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:18.100073 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448388e-a1fe-4ef8-ac0c-df3528bd97a9 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:18.100356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448388e-a1fe-4ef8-ac0c-df3528bd97a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:18.100694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448388e-a1fe-4ef8-ac0c-df3528bd97a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:18.101735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448388e-a1fe-4ef8-ac0c-df3528bd97a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:18.104183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448388e-a1fe-4ef8-ac0c-df3528bd97a9 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:18.192706 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 871/1764] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:18 2022] POST /identity/v3/projects => generated 391 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:18.193737 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448388e-a1fe-4ef8-ac0c-df3528bd97a9 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:18.197198 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec85ab-6656-418d-9ccb-c1de28ed30f0 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:18.282776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:18.330162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:18.332337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448388e-a1fe-4ef8-ac0c-df3528bd97a9 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:18.356967 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350afd6-423b-4eb4-a101-3e57cf1939db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:18.359614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350afd6-423b-4eb4-a101-3e57cf1939db admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:18.359959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350afd6-423b-4eb4-a101-3e57cf1939db admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:18.360299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350afd6-423b-4eb4-a101-3e57cf1939db admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:18.361546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350afd6-423b-4eb4-a101-3e57cf1939db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:18.364229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350afd6-423b-4eb4-a101-3e57cf1939db admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:18.380772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aece3c-0ef2-40a2-a342-5be9363f3dbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:18.382967 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aece3c-0ef2-40a2-a342-5be9363f3dbc service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:18.383361 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aece3c-0ef2-40a2-a342-5be9363f3dbc service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:18.383768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aece3c-0ef2-40a2-a342-5be9363f3dbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:18.393698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-93aece3c-0ef2-40a2-a342-5be9363f3dbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:18.396699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 872/1765] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:18 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:18.398765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350afd6-423b-4eb4-a101-3e57cf1939db admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:18.410846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350afd6-423b-4eb4-a101-3e57cf1939db admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:18.416960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f979dcc-1518-4767-ad07-2baf66beec57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:18.417788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f979dcc-1518-4767-ad07-2baf66beec57 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:18.417960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aece3c-0ef2-40a2-a342-5be9363f3dbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:18.418268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f979dcc-1518-4767-ad07-2baf66beec57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:18.418752 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f979dcc-1518-4767-ad07-2baf66beec57 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:18.420208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f979dcc-1518-4767-ad07-2baf66beec57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:18.423024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aece3c-0ef2-40a2-a342-5be9363f3dbc service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:18.429981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f979dcc-1518-4767-ad07-2baf66beec57 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:18.435017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 894/1766] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:18.452858 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 873/1767] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:18.454014 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aece3c-0ef2-40a2-a342-5be9363f3dbc service nova] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:18.467487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aece3c-0ef2-40a2-a342-5be9363f3dbc service nova] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:18.489567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f8b78-d4c7-4a32-b7d6-d7fead8baedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:18.490270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f8b78-d4c7-4a32-b7d6-d7fead8baedb admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:18.490558 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f8b78-d4c7-4a32-b7d6-d7fead8baedb admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:18.490841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f8b78-d4c7-4a32-b7d6-d7fead8baedb admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:18.494731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f8b78-d4c7-4a32-b7d6-d7fead8baedb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:18.498584 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f8b78-d4c7-4a32-b7d6-d7fead8baedb admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:18.520867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 895/1768] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:18.522020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f979dcc-1518-4767-ad07-2baf66beec57 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:18.536980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f979dcc-1518-4767-ad07-2baf66beec57 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:18.553243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f8b78-d4c7-4a32-b7d6-d7fead8baedb admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:18.556252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-aa4f8b78-d4c7-4a32-b7d6-d7fead8baedb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:18.563306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316d8ff-2678-480c-8695-d4b78c368aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:18.564224 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316d8ff-2678-480c-8695-d4b78c368aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:18.564563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316d8ff-2678-480c-8695-d4b78c368aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:18.564991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316d8ff-2678-480c-8695-d4b78c368aa8 admin admin] PATH_INFO: `/v3/projects/2f624edca273431798ca934aef196908/users/8b7d9d692bc84388b52bf74ef8011a74/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:18.580502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f8b78-d4c7-4a32-b7d6-d7fead8baedb admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:18.589870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbca76a-34de-460a-80e5-bc3ee6bbf395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:18.597337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316d8ff-2678-480c-8695-d4b78c368aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f624edca273431798ca934aef196908, user_id=8b7d9d692bc84388b52bf74ef8011a74, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:18.598139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbca76a-34de-460a-80e5-bc3ee6bbf395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:18.598660 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbca76a-34de-460a-80e5-bc3ee6bbf395 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:18.599359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbca76a-34de-460a-80e5-bc3ee6bbf395 admin admin] PATH_INFO: `/v3/users/9b5d000e3f5545e69944cf9e87cd8dad` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:18.600884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316d8ff-2678-480c-8695-d4b78c368aa8 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:18.601650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbca76a-34de-460a-80e5-bc3ee6bbf395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b5d000e3f5545e69944cf9e87cd8dad)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:18.604937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbca76a-34de-460a-80e5-bc3ee6bbf395 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:18.635372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 874/1769] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:18 2022] PUT /identity/v3/projects/2f624edca273431798ca934aef196908/users/8b7d9d692bc84388b52bf74ef8011a74/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:18.646728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177466d4-1c14-42f0-9271-99085043153f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:18.647370 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177466d4-1c14-42f0-9271-99085043153f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:18.648356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177466d4-1c14-42f0-9271-99085043153f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:18.706250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-177466d4-1c14-42f0-9271-99085043153f None None] MFA Rules not processed for user `8b7d9d692bc84388b52bf74ef8011a74`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:18.714052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-177466d4-1c14-42f0-9271-99085043153f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:18.770781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2dbca76a-34de-460a-80e5-bc3ee6bbf395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b5d000e3f5545e69944cf9e87cd8dad'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:18.771666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2dbca76a-34de-460a-80e5-bc3ee6bbf395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b5d000e3f5545e69944cf9e87cd8dad'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:18.803686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2dbca76a-34de-460a-80e5-bc3ee6bbf395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b5d000e3f5545e69944cf9e87cd8dad'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:18.841217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2dbca76a-34de-460a-80e5-bc3ee6bbf395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b5d000e3f5545e69944cf9e87cd8dad'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:18.852462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2dbca76a-34de-460a-80e5-bc3ee6bbf395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b5d000e3f5545e69944cf9e87cd8dad'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:18.874721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 896/1770] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:18 2022] DELETE /identity/v3/users/9b5d000e3f5545e69944cf9e87cd8dad => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:18.880602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c1d72-b6c9-46b0-aca8-bee602c434da None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:18.881726 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c1d72-b6c9-46b0-aca8-bee602c434da None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:18.882883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c1d72-b6c9-46b0-aca8-bee602c434da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:18.885850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 875/1771] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:18.894169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177466d4-1c14-42f0-9271-99085043153f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:18.919294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-911c1d72-b6c9-46b0-aca8-bee602c434da None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:18.921110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-911c1d72-b6c9-46b0-aca8-bee602c434da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:19.018122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177466d4-1c14-42f0-9271-99085043153f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:19.023528 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 897/1772] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:19.028692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd9fe0-611c-4ee0-8079-a69b9d02805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:19.029647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd9fe0-611c-4ee0-8079-a69b9d02805d admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:19.030042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd9fe0-611c-4ee0-8079-a69b9d02805d admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:19.030587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd9fe0-611c-4ee0-8079-a69b9d02805d admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:19.031485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd9fe0-611c-4ee0-8079-a69b9d02805d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:19.034372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c1d72-b6c9-46b0-aca8-bee602c434da None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:19.035788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-911c1d72-b6c9-46b0-aca8-bee602c434da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:19.036350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd9fe0-611c-4ee0-8079-a69b9d02805d admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:19.089861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:19.100671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 876/1773] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:18 2022] POST /identity/v3/projects => generated 391 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:19.102967 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd9fe0-611c-4ee0-8079-a69b9d02805d admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:19.105001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c1fd9fe0-611c-4ee0-8079-a69b9d02805d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:19.114685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:19.117776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c1d72-b6c9-46b0-aca8-bee602c434da None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:19.171527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd3885-1520-4a0a-aff4-b506d036d4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:19.172235 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd3885-1520-4a0a-aff4-b506d036d4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:19.172592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd3885-1520-4a0a-aff4-b506d036d4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:19.172870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd3885-1520-4a0a-aff4-b506d036d4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:19.182571 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd3885-1520-4a0a-aff4-b506d036d4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:19.184024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd3885-1520-4a0a-aff4-b506d036d4a1 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:19.184743 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd9fe0-611c-4ee0-8079-a69b9d02805d admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:19.189812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f643f-fece-4d6e-b98e-e76af81ce0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:19.190355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f643f-fece-4d6e-b98e-e76af81ce0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:19.195265 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 898/1774] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:19.196795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f643f-fece-4d6e-b98e-e76af81ce0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:19.197184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f643f-fece-4d6e-b98e-e76af81ce0e4 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:19.198015 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f643f-fece-4d6e-b98e-e76af81ce0e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:19.200027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f643f-fece-4d6e-b98e-e76af81ce0e4 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:19.221736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 877/1775] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:19 2022] POST /identity/v3/users => generated 302 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:19.225909 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd3885-1520-4a0a-aff4-b506d036d4a1 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:19.236890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd3885-1520-4a0a-aff4-b506d036d4a1 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:19.242048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0cb43-2c2c-4d28-b594-604beaeb3398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:19.242798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0cb43-2c2c-4d28-b594-604beaeb3398 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:19.243230 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0cb43-2c2c-4d28-b594-604beaeb3398 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:19.243513 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0cb43-2c2c-4d28-b594-604beaeb3398 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:19.244429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0cb43-2c2c-4d28-b594-604beaeb3398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:19.246297 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0cb43-2c2c-4d28-b594-604beaeb3398 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:19.273431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 899/1776] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:19.277961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f643f-fece-4d6e-b98e-e76af81ce0e4 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:19.290457 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f643f-fece-4d6e-b98e-e76af81ce0e4 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:19.301075 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1feb4b0-2dfb-4310-91df-29ecc91333b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:19.302016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1feb4b0-2dfb-4310-91df-29ecc91333b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:19.302468 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1feb4b0-2dfb-4310-91df-29ecc91333b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:19.302805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1feb4b0-2dfb-4310-91df-29ecc91333b0 admin admin] PATH_INFO: `/v3/projects/91e54e396e2c48ed9b407c2966a6a84b/users/43fdc49753e64a05889bb7da8ab76235/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:19.322399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1feb4b0-2dfb-4310-91df-29ecc91333b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e54e396e2c48ed9b407c2966a6a84b, user_id=43fdc49753e64a05889bb7da8ab76235, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:19.325127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1feb4b0-2dfb-4310-91df-29ecc91333b0 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:19.339376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 878/1777] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:19 2022] PUT /identity/v3/projects/91e54e396e2c48ed9b407c2966a6a84b/users/43fdc49753e64a05889bb7da8ab76235/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:19.345518 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0cb43-2c2c-4d28-b594-604beaeb3398 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:19.356986 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0cb43-2c2c-4d28-b594-604beaeb3398 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:19.417637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3df0d6-dc11-4f44-b58a-3c9c3392c53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:19.418361 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df0d6-dc11-4f44-b58a-3c9c3392c53b admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:19.418654 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df0d6-dc11-4f44-b58a-3c9c3392c53b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:19.418934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df0d6-dc11-4f44-b58a-3c9c3392c53b admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:19.419703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3df0d6-dc11-4f44-b58a-3c9c3392c53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:19.421831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3df0d6-dc11-4f44-b58a-3c9c3392c53b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:19.432709 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 900/1778] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:19.437598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1feb4b0-2dfb-4310-91df-29ecc91333b0 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:19.447935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1feb4b0-2dfb-4310-91df-29ecc91333b0 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:19.452630 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be9471-044d-4553-89e2-01ece79621ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:19.453723 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be9471-044d-4553-89e2-01ece79621ca admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:19.453992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be9471-044d-4553-89e2-01ece79621ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:19.454322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be9471-044d-4553-89e2-01ece79621ca admin admin] PATH_INFO: `/v3/projects/91e54e396e2c48ed9b407c2966a6a84b/users/43fdc49753e64a05889bb7da8ab76235/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:19.456515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be9471-044d-4553-89e2-01ece79621ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e54e396e2c48ed9b407c2966a6a84b, user_id=43fdc49753e64a05889bb7da8ab76235, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:19.459231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be9471-044d-4553-89e2-01ece79621ca admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:19.471087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 879/1779] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:19 2022] PUT /identity/v3/projects/91e54e396e2c48ed9b407c2966a6a84b/users/43fdc49753e64a05889bb7da8ab76235/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:19.478685 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3df0d6-dc11-4f44-b58a-3c9c3392c53b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:19.480424 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ad3df0d6-dc11-4f44-b58a-3c9c3392c53b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:19.482207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c166b49-e5e6-4515-b9bf-3a0c79c970cd None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:19.482722 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c166b49-e5e6-4515-b9bf-3a0c79c970cd None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:19.483174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c166b49-e5e6-4515-b9bf-3a0c79c970cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:19.494266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3df0d6-dc11-4f44-b58a-3c9c3392c53b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:19.533652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-0c166b49-e5e6-4515-b9bf-3a0c79c970cd None None] MFA Rules not processed for user `43fdc49753e64a05889bb7da8ab76235`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:19.534865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0c166b49-e5e6-4515-b9bf-3a0c79c970cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:19.567769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:19.569598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:19.569888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:19.571167 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] PATH_INFO: `/v3/projects/c1790fa30d5d4d9485699dd95e188337` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:19.572971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1790fa30d5d4d9485699dd95e188337)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:19.574905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:19.601033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:19.613746 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 880/1780] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:19.625563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c166b49-e5e6-4515-b9bf-3a0c79c970cd None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:19.627071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0c166b49-e5e6-4515-b9bf-3a0c79c970cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:19.639543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1790fa30d5d4d9485699dd95e188337', 'request_id': 'req-7898195c-b901-4f98-b798-e19abcd9d8fd'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:19.649399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1790fa30d5d4d9485699dd95e188337', 'request_id': 'req-7898195c-b901-4f98-b798-e19abcd9d8fd'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:19.656498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1790fa30d5d4d9485699dd95e188337', 'request_id': 'req-7898195c-b901-4f98-b798-e19abcd9d8fd'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:19.680083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] The token cache is being invalidate because project c1790fa30d5d4d9485699dd95e188337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:19.680379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:19.681365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:19.684000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:19.684147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 901/1781] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:19 2022] DELETE /identity/v3/projects/c1790fa30d5d4d9485699dd95e188337 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:19.687820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:19.689291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:19.695667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c166b49-e5e6-4515-b9bf-3a0c79c970cd None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:19.755086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dca8e8-b1e5-403b-820e-d60932229b75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:19.755756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dca8e8-b1e5-403b-820e-d60932229b75 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:19.756078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dca8e8-b1e5-403b-820e-d60932229b75 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:19.756366 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dca8e8-b1e5-403b-820e-d60932229b75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:19.758973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c3dca8e8-b1e5-403b-820e-d60932229b75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:19.762516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898195c-b901-4f98-b798-e19abcd9d8fd admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:19.767407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:19.768353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:19.768603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:19.768940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] PATH_INFO: `/v3/users/c79ccca2efdf42369224bf0e9b80aed2` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:19.770325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c79ccca2efdf42369224bf0e9b80aed2)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:19.772474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:19.821003 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c79ccca2efdf42369224bf0e9b80aed2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:19.821835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c79ccca2efdf42369224bf0e9b80aed2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:19.827696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dca8e8-b1e5-403b-820e-d60932229b75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:19.829286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dca8e8-b1e5-403b-820e-d60932229b75 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:19.829418 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c79ccca2efdf42369224bf0e9b80aed2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:19.839116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 881/1782] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:46:19.841470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c79ccca2efdf42369224bf0e9b80aed2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:19.849358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c79ccca2efdf42369224bf0e9b80aed2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:19.855573 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 902/1783] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:19 2022] DELETE /identity/v3/users/c79ccca2efdf42369224bf0e9b80aed2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:19.899914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dca8e8-b1e5-403b-820e-d60932229b75 service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:19.909148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c3dca8e8-b1e5-403b-820e-d60932229b75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:19.912797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dca8e8-b1e5-403b-820e-d60932229b75 service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:19.984144 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c16ff4-747d-4cdb-b045-469b1e341b1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:19.984979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c16ff4-747d-4cdb-b045-469b1e341b1b service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:19.985274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c16ff4-747d-4cdb-b045-469b1e341b1b service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:19.985553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c16ff4-747d-4cdb-b045-469b1e341b1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:19.987623 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-88c16ff4-747d-4cdb-b045-469b1e341b1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:20.057279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c16ff4-747d-4cdb-b045-469b1e341b1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:20.058089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c16ff4-747d-4cdb-b045-469b1e341b1b service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:20.069824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 882/1784] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:20.206718 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:20.208836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:20.262773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:20.341029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ec152-ce5a-4b96-9fe5-8266b455bcd6 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:20.347093 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:20.349423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:20.349730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:20.350069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd admin admin] PATH_INFO: `/v3/projects/4c4dcf774e514b88b27042534ad02fd1` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:20.351693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c4dcf774e514b88b27042534ad02fd1)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:20.353841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:20.430980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c4dcf774e514b88b27042534ad02fd1', 'request_id': 'req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:20.446920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c4dcf774e514b88b27042534ad02fd1', 'request_id': 'req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:20.454273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c4dcf774e514b88b27042534ad02fd1', 'request_id': 'req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:20.468246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd admin admin] The token cache is being invalidate because project 4c4dcf774e514b88b27042534ad02fd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:20.468602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:20.469516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8aa7ac7c-f956-4a68-92fd-c4cd3ae4a4bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:20.471217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 903/1785] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:20 2022] DELETE /identity/v3/projects/4c4dcf774e514b88b27042534ad02fd1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:20.476172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a96cb-6052-4a91-8d81-110788286be2 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:20.476602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a96cb-6052-4a91-8d81-110788286be2 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:20.477093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a96cb-6052-4a91-8d81-110788286be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:20.485228 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:20.502338 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-820a96cb-6052-4a91-8d81-110788286be2 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:20.512834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-820a96cb-6052-4a91-8d81-110788286be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:20.603729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 883/1786] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:20.612680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f52250-a336-44f8-8906-ec689bb71415 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:20.613205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f52250-a336-44f8-8906-ec689bb71415 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:20.613436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f52250-a336-44f8-8906-ec689bb71415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:20.637271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-92f52250-a336-44f8-8906-ec689bb71415 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:20.638603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-92f52250-a336-44f8-8906-ec689bb71415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:20.648015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 904/1787] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:20.652261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a96cb-6052-4a91-8d81-110788286be2 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:20.663658 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a96cb-6052-4a91-8d81-110788286be2 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:20.668194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612b7f0-22cb-499d-b049-6f539d5696ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:20.668891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b7f0-22cb-499d-b049-6f539d5696ec admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:20.669181 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b7f0-22cb-499d-b049-6f539d5696ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:20.669458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b7f0-22cb-499d-b049-6f539d5696ec admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:20.670189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612b7f0-22cb-499d-b049-6f539d5696ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:20.670500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612b7f0-22cb-499d-b049-6f539d5696ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:20.672137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612b7f0-22cb-499d-b049-6f539d5696ec admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:20.684708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 884/1788] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:46:20.689218 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f52250-a336-44f8-8906-ec689bb71415 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:20.706606 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f52250-a336-44f8-8906-ec689bb71415 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:20.711270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975599ef-492a-4972-9721-3fc6b01a7ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:20.711930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975599ef-492a-4972-9721-3fc6b01a7ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:20.712213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975599ef-492a-4972-9721-3fc6b01a7ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:20.712488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975599ef-492a-4972-9721-3fc6b01a7ef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:20.713231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975599ef-492a-4972-9721-3fc6b01a7ef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:20.715126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975599ef-492a-4972-9721-3fc6b01a7ef0 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:20.744706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 905/1789] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:20 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:20.749085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612b7f0-22cb-499d-b049-6f539d5696ec admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:20.761185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612b7f0-22cb-499d-b049-6f539d5696ec admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:20.830927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ba601-7f15-4cb2-afa8-b9499bfdaf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:20.831614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ba601-7f15-4cb2-afa8-b9499bfdaf38 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:20.831904 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ba601-7f15-4cb2-afa8-b9499bfdaf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:20.832184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ba601-7f15-4cb2-afa8-b9499bfdaf38 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:20.832936 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ba601-7f15-4cb2-afa8-b9499bfdaf38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:20.834907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ba601-7f15-4cb2-afa8-b9499bfdaf38 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:20.862168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 885/1790] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:20 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:20.863383 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975599ef-492a-4972-9721-3fc6b01a7ef0 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:20.875472 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975599ef-492a-4972-9721-3fc6b01a7ef0 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:20.880117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cf308-ec2a-4bd4-9ad6-577f231b8902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:20.880826 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cf308-ec2a-4bd4-9ad6-577f231b8902 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:20.881111 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cf308-ec2a-4bd4-9ad6-577f231b8902 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:20.881389 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cf308-ec2a-4bd4-9ad6-577f231b8902 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:20.882118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cf308-ec2a-4bd4-9ad6-577f231b8902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:20.883692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cf308-ec2a-4bd4-9ad6-577f231b8902 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:20.894700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 906/1791] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:20.898610 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ba601-7f15-4cb2-afa8-b9499bfdaf38 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:20.908803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ba601-7f15-4cb2-afa8-b9499bfdaf38 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:20.914272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4c418-b5f1-416d-bec2-480df55d1bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:20.914949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4c418-b5f1-416d-bec2-480df55d1bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:20.915228 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4c418-b5f1-416d-bec2-480df55d1bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:20.915501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4c418-b5f1-416d-bec2-480df55d1bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:20.916231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4c418-b5f1-416d-bec2-480df55d1bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:20.918484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4c418-b5f1-416d-bec2-480df55d1bb6 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:20.929785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 886/1792] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:20.933884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cf308-ec2a-4bd4-9ad6-577f231b8902 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:20.944313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cf308-ec2a-4bd4-9ad6-577f231b8902 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:20.948842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee2607-adbd-49ed-a295-e5c0b04b1568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:20.949713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee2607-adbd-49ed-a295-e5c0b04b1568 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:20.949996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee2607-adbd-49ed-a295-e5c0b04b1568 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:20.950271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee2607-adbd-49ed-a295-e5c0b04b1568 admin admin] PATH_INFO: `/v3/projects/ff717f31db2b48e8baaf98ebdd214db7/users/f6443d29643f4c0693d6f7e3b7454257/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:20.964317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee2607-adbd-49ed-a295-e5c0b04b1568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff717f31db2b48e8baaf98ebdd214db7, user_id=f6443d29643f4c0693d6f7e3b7454257, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:20.967002 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee2607-adbd-49ed-a295-e5c0b04b1568 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:20.980259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 907/1793] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:20 2022] PUT /identity/v3/projects/ff717f31db2b48e8baaf98ebdd214db7/users/f6443d29643f4c0693d6f7e3b7454257/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:20.990868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101bf9c-384a-4106-aa11-3f780ce1e417 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:20.991167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101bf9c-384a-4106-aa11-3f780ce1e417 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:20.991484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101bf9c-384a-4106-aa11-3f780ce1e417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:21.020760 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-f101bf9c-384a-4106-aa11-3f780ce1e417 None None] MFA Rules not processed for user `f6443d29643f4c0693d6f7e3b7454257`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:21.022149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f101bf9c-384a-4106-aa11-3f780ce1e417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:21.084263 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:21.090721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 887/1794] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:21.095563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee2607-adbd-49ed-a295-e5c0b04b1568 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:21.147870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:21.160489 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee2607-adbd-49ed-a295-e5c0b04b1568 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:21.165529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcc98f-9c9a-4ebf-9b01-6acd34963df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:21.166189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcc98f-9c9a-4ebf-9b01-6acd34963df9 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:21.166470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcc98f-9c9a-4ebf-9b01-6acd34963df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:21.166744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcc98f-9c9a-4ebf-9b01-6acd34963df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:21.167468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcc98f-9c9a-4ebf-9b01-6acd34963df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:21.169456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcc98f-9c9a-4ebf-9b01-6acd34963df9 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:21.196455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 908/1795] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:21 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:21.200587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101bf9c-384a-4106-aa11-3f780ce1e417 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:21.212063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101bf9c-384a-4106-aa11-3f780ce1e417 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:21.266301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565212b-3c64-4584-9fcb-d51d6d1052b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:21.266959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565212b-3c64-4584-9fcb-d51d6d1052b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:21.267239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565212b-3c64-4584-9fcb-d51d6d1052b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:21.267512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565212b-3c64-4584-9fcb-d51d6d1052b7 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:21.268208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565212b-3c64-4584-9fcb-d51d6d1052b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:21.270153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565212b-3c64-4584-9fcb-d51d6d1052b7 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:21.293013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 888/1796] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:21 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:21.294357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcc98f-9c9a-4ebf-9b01-6acd34963df9 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:21.305681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcc98f-9c9a-4ebf-9b01-6acd34963df9 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:21.313145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09f20-bef5-499a-91d0-a4834a8360ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:21.313819 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09f20-bef5-499a-91d0-a4834a8360ec admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:21.314108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09f20-bef5-499a-91d0-a4834a8360ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:21.314391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09f20-bef5-499a-91d0-a4834a8360ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:21.315132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe09f20-bef5-499a-91d0-a4834a8360ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:21.316760 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe09f20-bef5-499a-91d0-a4834a8360ec admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:21.328197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 909/1797] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:21.332624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565212b-3c64-4584-9fcb-d51d6d1052b7 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:21.342467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565212b-3c64-4584-9fcb-d51d6d1052b7 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:21.347077 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106879b0-119a-4e35-841f-d1d5f3ca3fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:21.347964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106879b0-119a-4e35-841f-d1d5f3ca3fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:21.348245 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106879b0-119a-4e35-841f-d1d5f3ca3fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:21.348518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106879b0-119a-4e35-841f-d1d5f3ca3fa3 admin admin] PATH_INFO: `/v3/projects/42347aca54a0418c998a4e6c4aebf1b7/users/098cdbec22be4de4ba5ac16a471fdac8/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:21.366570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106879b0-119a-4e35-841f-d1d5f3ca3fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42347aca54a0418c998a4e6c4aebf1b7, user_id=098cdbec22be4de4ba5ac16a471fdac8, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:21.369907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106879b0-119a-4e35-841f-d1d5f3ca3fa3 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:21.394954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 889/1798] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:21 2022] PUT /identity/v3/projects/42347aca54a0418c998a4e6c4aebf1b7/users/098cdbec22be4de4ba5ac16a471fdac8/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:21.395568 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09f20-bef5-499a-91d0-a4834a8360ec admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:21.407160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09f20-bef5-499a-91d0-a4834a8360ec admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:21.469542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55dca27-4896-4593-b8b6-3f34e8b54434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:21.470259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55dca27-4896-4593-b8b6-3f34e8b54434 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:21.470506 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55dca27-4896-4593-b8b6-3f34e8b54434 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:21.470784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55dca27-4896-4593-b8b6-3f34e8b54434 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:21.471517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55dca27-4896-4593-b8b6-3f34e8b54434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:21.473134 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55dca27-4896-4593-b8b6-3f34e8b54434 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:21.486457 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 910/1799] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:21.491273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106879b0-119a-4e35-841f-d1d5f3ca3fa3 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:21.527331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106879b0-119a-4e35-841f-d1d5f3ca3fa3 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:21.533213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05723a3-0a90-4c2e-8a1e-513523be82cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:21.534136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05723a3-0a90-4c2e-8a1e-513523be82cd admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:21.534491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05723a3-0a90-4c2e-8a1e-513523be82cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:21.534910 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05723a3-0a90-4c2e-8a1e-513523be82cd admin admin] PATH_INFO: `/v3/projects/42347aca54a0418c998a4e6c4aebf1b7/users/098cdbec22be4de4ba5ac16a471fdac8/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:21.539259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05723a3-0a90-4c2e-8a1e-513523be82cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=42347aca54a0418c998a4e6c4aebf1b7, user_id=098cdbec22be4de4ba5ac16a471fdac8, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:21.540076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05723a3-0a90-4c2e-8a1e-513523be82cd admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:21.555910 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 890/1800] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:21 2022] PUT /identity/v3/projects/42347aca54a0418c998a4e6c4aebf1b7/users/098cdbec22be4de4ba5ac16a471fdac8/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:21.567857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd1269-738c-496e-a36e-b2d312e22d9a None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:21.568332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd1269-738c-496e-a36e-b2d312e22d9a None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:21.568697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd1269-738c-496e-a36e-b2d312e22d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:21.606260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-ecdd1269-738c-496e-a36e-b2d312e22d9a None None] MFA Rules not processed for user `098cdbec22be4de4ba5ac16a471fdac8`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:21.607878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ecdd1269-738c-496e-a36e-b2d312e22d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:21.704946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 911/1801] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:21.716653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05723a3-0a90-4c2e-8a1e-513523be82cd admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:21.719298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c05723a3-0a90-4c2e-8a1e-513523be82cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:21.787703 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05723a3-0a90-4c2e-8a1e-513523be82cd admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:21.791897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca4759-4026-44b1-89c9-2712bddc6774 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:21.792610 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca4759-4026-44b1-89c9-2712bddc6774 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:21.793006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca4759-4026-44b1-89c9-2712bddc6774 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:21.793132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca4759-4026-44b1-89c9-2712bddc6774 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:21.804088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca4759-4026-44b1-89c9-2712bddc6774 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:21.805871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca4759-4026-44b1-89c9-2712bddc6774 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:21.829715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 891/1802] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:21.849955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd1269-738c-496e-a36e-b2d312e22d9a None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:21.938316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:21.958934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd1269-738c-496e-a36e-b2d312e22d9a None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:21.963599 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:21.964483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:21.964910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:21.965193 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] PATH_INFO: `/v3/users/f6443d29643f4c0693d6f7e3b7454257` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:21.966404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6443d29643f4c0693d6f7e3b7454257)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:21.969085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:22.017283 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6443d29643f4c0693d6f7e3b7454257'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.017951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6443d29643f4c0693d6f7e3b7454257'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.026822 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6443d29643f4c0693d6f7e3b7454257'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.039904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6443d29643f4c0693d6f7e3b7454257'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.047319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6443d29643f4c0693d6f7e3b7454257'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.053507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 912/1803] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:21 2022] DELETE /identity/v3/users/f6443d29643f4c0693d6f7e3b7454257 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:22.059553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f3ac6-427f-40b2-bd83-8a11452acbec None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:22.059553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f3ac6-427f-40b2-bd83-8a11452acbec None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:22.059553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f3ac6-427f-40b2-bd83-8a11452acbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:22.078441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-105f3ac6-427f-40b2-bd83-8a11452acbec None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:22.079654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-105f3ac6-427f-40b2-bd83-8a11452acbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:22.128665 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:22.143286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 892/1804] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:22.153153 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:22.154633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:22.218718 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a641370-6706-4322-b339-ed279090b15c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:22.222384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5f671-26bb-48d1-8180-29d84dca0c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:22.223053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5f671-26bb-48d1-8180-29d84dca0c46 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:22.223341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5f671-26bb-48d1-8180-29d84dca0c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:22.223624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5f671-26bb-48d1-8180-29d84dca0c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:22.234387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5f671-26bb-48d1-8180-29d84dca0c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:22.235851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5f671-26bb-48d1-8180-29d84dca0c46 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:22.246495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 913/1805] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:22.540840 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f3ac6-427f-40b2-bd83-8a11452acbec None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:22.542370 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-105f3ac6-427f-40b2-bd83-8a11452acbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:22.561326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f3ac6-427f-40b2-bd83-8a11452acbec None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:22.567716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:22.568801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:22.569109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:22.569477 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] PATH_INFO: `/v3/projects/ff717f31db2b48e8baaf98ebdd214db7` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:22.570593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff717f31db2b48e8baaf98ebdd214db7)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:22.572497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:22.651751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff717f31db2b48e8baaf98ebdd214db7', 'request_id': 'req-41d762a3-2c22-4976-b1a1-bc7026943faa'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.663069 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff717f31db2b48e8baaf98ebdd214db7', 'request_id': 'req-41d762a3-2c22-4976-b1a1-bc7026943faa'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.669815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff717f31db2b48e8baaf98ebdd214db7', 'request_id': 'req-41d762a3-2c22-4976-b1a1-bc7026943faa'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.687298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] The token cache is being invalidate because project ff717f31db2b48e8baaf98ebdd214db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:22.687600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.688281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.690087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 893/1806] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:22 2022] DELETE /identity/v3/projects/ff717f31db2b48e8baaf98ebdd214db7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:22.693928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5f671-26bb-48d1-8180-29d84dca0c46 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:22.704858 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5f671-26bb-48d1-8180-29d84dca0c46 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:22.779913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b559ea-d79e-4342-833b-68a976bd031c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:22.780853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b559ea-d79e-4342-833b-68a976bd031c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:22.781139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b559ea-d79e-4342-833b-68a976bd031c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:22.781417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b559ea-d79e-4342-833b-68a976bd031c admin admin] PATH_INFO: `/v3/users/098cdbec22be4de4ba5ac16a471fdac8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:22.782641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b559ea-d79e-4342-833b-68a976bd031c admin admin] RBAC: Authorizing `identity:delete_user(user_id=098cdbec22be4de4ba5ac16a471fdac8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:22.784565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b559ea-d79e-4342-833b-68a976bd031c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:22.800732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:22.846377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-78b559ea-d79e-4342-833b-68a976bd031c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '098cdbec22be4de4ba5ac16a471fdac8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.847160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-78b559ea-d79e-4342-833b-68a976bd031c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '098cdbec22be4de4ba5ac16a471fdac8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.855687 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-78b559ea-d79e-4342-833b-68a976bd031c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '098cdbec22be4de4ba5ac16a471fdac8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.869648 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-78b559ea-d79e-4342-833b-68a976bd031c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '098cdbec22be4de4ba5ac16a471fdac8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.877174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-78b559ea-d79e-4342-833b-68a976bd031c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '098cdbec22be4de4ba5ac16a471fdac8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:22.884828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 914/1807] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:22 2022] DELETE /identity/v3/users/098cdbec22be4de4ba5ac16a471fdac8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:23.197842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:23.200212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:23.214312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d762a3-2c22-4976-b1a1-bc7026943faa admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:23.275143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcc755-3cfa-4f59-8412-c1e08e3822bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:23.276018 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcc755-3cfa-4f59-8412-c1e08e3822bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:23.276301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcc755-3cfa-4f59-8412-c1e08e3822bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:23.276591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcc755-3cfa-4f59-8412-c1e08e3822bb admin admin] PATH_INFO: `/v3/projects/42347aca54a0418c998a4e6c4aebf1b7` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:23.277803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bcc755-3cfa-4f59-8412-c1e08e3822bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=42347aca54a0418c998a4e6c4aebf1b7)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:23.279778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bcc755-3cfa-4f59-8412-c1e08e3822bb admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:23.328764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-34bcc755-3cfa-4f59-8412-c1e08e3822bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42347aca54a0418c998a4e6c4aebf1b7', 'request_id': 'req-34bcc755-3cfa-4f59-8412-c1e08e3822bb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:23.337980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-34bcc755-3cfa-4f59-8412-c1e08e3822bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42347aca54a0418c998a4e6c4aebf1b7', 'request_id': 'req-34bcc755-3cfa-4f59-8412-c1e08e3822bb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:23.351935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-34bcc755-3cfa-4f59-8412-c1e08e3822bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42347aca54a0418c998a4e6c4aebf1b7', 'request_id': 'req-34bcc755-3cfa-4f59-8412-c1e08e3822bb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:23.363440 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-34bcc755-3cfa-4f59-8412-c1e08e3822bb admin admin] The token cache is being invalidate because project 42347aca54a0418c998a4e6c4aebf1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:23.363729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-34bcc755-3cfa-4f59-8412-c1e08e3822bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:23.364619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-34bcc755-3cfa-4f59-8412-c1e08e3822bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:23.366177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 894/1808] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:23 2022] DELETE /identity/v3/projects/42347aca54a0418c998a4e6c4aebf1b7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:23.370995 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fbaa4-49a8-437c-9204-a674c10264e1 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:23.371289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fbaa4-49a8-437c-9204-a674c10264e1 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:23.371567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fbaa4-49a8-437c-9204-a674c10264e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:23.391869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-464fbaa4-49a8-437c-9204-a674c10264e1 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:23.393118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-464fbaa4-49a8-437c-9204-a674c10264e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:23.453059 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 915/1809] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:23.460772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b06525-4a55-471a-8b8f-604341b9edbc None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:23.461158 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b06525-4a55-471a-8b8f-604341b9edbc None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:23.461312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b06525-4a55-471a-8b8f-604341b9edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:23.478008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:23.484317 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-89b06525-4a55-471a-8b8f-604341b9edbc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:23.486305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-89b06525-4a55-471a-8b8f-604341b9edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:23.495779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 895/1810] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:23.500123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fbaa4-49a8-437c-9204-a674c10264e1 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:23.509871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fbaa4-49a8-437c-9204-a674c10264e1 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:23.514671 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128340b-999e-4dca-a3bf-3be09bd397a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:23.515323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128340b-999e-4dca-a3bf-3be09bd397a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:23.515604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128340b-999e-4dca-a3bf-3be09bd397a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:23.515879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128340b-999e-4dca-a3bf-3be09bd397a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:23.516674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128340b-999e-4dca-a3bf-3be09bd397a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:23.516977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128340b-999e-4dca-a3bf-3be09bd397a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:23.518559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128340b-999e-4dca-a3bf-3be09bd397a4 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:23.532642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 916/1811] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:46:23.534809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b06525-4a55-471a-8b8f-604341b9edbc None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:23.544000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b06525-4a55-471a-8b8f-604341b9edbc None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:23.549545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79c18b-5395-4678-85f6-f23556aff264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:23.550198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79c18b-5395-4678-85f6-f23556aff264 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:23.550482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79c18b-5395-4678-85f6-f23556aff264 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:23.550758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79c18b-5395-4678-85f6-f23556aff264 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:23.551478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79c18b-5395-4678-85f6-f23556aff264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:23.553402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79c18b-5395-4678-85f6-f23556aff264 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:23.585821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 896/1812] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:23 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:23.586959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128340b-999e-4dca-a3bf-3be09bd397a4 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:23.597651 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128340b-999e-4dca-a3bf-3be09bd397a4 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:23.657449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea23d79-e17f-4010-a4ca-c50ce1beefda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:23.658134 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea23d79-e17f-4010-a4ca-c50ce1beefda admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:23.658426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea23d79-e17f-4010-a4ca-c50ce1beefda admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:23.658724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea23d79-e17f-4010-a4ca-c50ce1beefda admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:23.659419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea23d79-e17f-4010-a4ca-c50ce1beefda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:23.661433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea23d79-e17f-4010-a4ca-c50ce1beefda admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:23.683187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 917/1813] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:23 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:23.687220 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79c18b-5395-4678-85f6-f23556aff264 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:23.697453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79c18b-5395-4678-85f6-f23556aff264 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:23.702517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc321d-f75c-498c-a7e3-bdf59a1e913b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:23.703168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc321d-f75c-498c-a7e3-bdf59a1e913b admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:23.703449 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc321d-f75c-498c-a7e3-bdf59a1e913b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:23.703720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc321d-f75c-498c-a7e3-bdf59a1e913b admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:23.704441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc321d-f75c-498c-a7e3-bdf59a1e913b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:23.706126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc321d-f75c-498c-a7e3-bdf59a1e913b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:23.715763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 897/1814] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:23.720245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea23d79-e17f-4010-a4ca-c50ce1beefda admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:23.730515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea23d79-e17f-4010-a4ca-c50ce1beefda admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:23.736419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b6def-12af-49a3-9abd-6143a3c13433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:23.737103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b6def-12af-49a3-9abd-6143a3c13433 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:23.737391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b6def-12af-49a3-9abd-6143a3c13433 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:23.737668 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b6def-12af-49a3-9abd-6143a3c13433 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:23.738407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b6def-12af-49a3-9abd-6143a3c13433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:23.740000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b6def-12af-49a3-9abd-6143a3c13433 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:23.741405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:23.752514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 918/1815] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:23.757102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc321d-f75c-498c-a7e3-bdf59a1e913b admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:23.779250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc321d-f75c-498c-a7e3-bdf59a1e913b admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:23.784156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1aab1-c775-40a3-ad48-724e5cd56b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:23.785080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1aab1-c775-40a3-ad48-724e5cd56b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:23.785365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1aab1-c775-40a3-ad48-724e5cd56b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:23.785640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1aab1-c775-40a3-ad48-724e5cd56b39 admin admin] PATH_INFO: `/v3/projects/8e6cb16388ca400e8f9e193a6b98df11/users/21b700175b514febba6cada0301ef194/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:23.801554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1aab1-c775-40a3-ad48-724e5cd56b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6cb16388ca400e8f9e193a6b98df11, user_id=21b700175b514febba6cada0301ef194, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:23.804242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1aab1-c775-40a3-ad48-724e5cd56b39 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:23.818865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 898/1816] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:23 2022] PUT /identity/v3/projects/8e6cb16388ca400e8f9e193a6b98df11/users/21b700175b514febba6cada0301ef194/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:23.829144 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7893a97-ed01-483b-b050-1d220e618790 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:23.829631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7893a97-ed01-483b-b050-1d220e618790 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:23.829718 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7893a97-ed01-483b-b050-1d220e618790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:23.857534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a7893a97-ed01-483b-b050-1d220e618790 None None] MFA Rules not processed for user `21b700175b514febba6cada0301ef194`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:23.859300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a7893a97-ed01-483b-b050-1d220e618790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:23.923423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 919/1817] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:23.927840 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1aab1-c775-40a3-ad48-724e5cd56b39 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:23.989544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1aab1-c775-40a3-ad48-724e5cd56b39 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:23.994196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8a28c-fb01-4fd8-9c6f-408fbc595183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:23.994854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8a28c-fb01-4fd8-9c6f-408fbc595183 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:23.995136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8a28c-fb01-4fd8-9c6f-408fbc595183 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:23.995414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8a28c-fb01-4fd8-9c6f-408fbc595183 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:23.996109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8a28c-fb01-4fd8-9c6f-408fbc595183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:23.998034 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8a28c-fb01-4fd8-9c6f-408fbc595183 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:24.029266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 899/1818] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:23 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:24.031322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7893a97-ed01-483b-b050-1d220e618790 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:24.042370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7893a97-ed01-483b-b050-1d220e618790 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:24.101263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe4b85-6b50-47b5-9981-b261e68cd1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:24.102245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe4b85-6b50-47b5-9981-b261e68cd1fd admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:24.102488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe4b85-6b50-47b5-9981-b261e68cd1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:24.102952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe4b85-6b50-47b5-9981-b261e68cd1fd admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:24.103970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe4b85-6b50-47b5-9981-b261e68cd1fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:24.106522 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe4b85-6b50-47b5-9981-b261e68cd1fd admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:24.129845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 920/1819] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:24 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:24.133879 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8a28c-fb01-4fd8-9c6f-408fbc595183 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:24.144756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8a28c-fb01-4fd8-9c6f-408fbc595183 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:24.149487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b523262-2aae-43c5-a358-6ec1624639d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:24.150158 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b523262-2aae-43c5-a358-6ec1624639d1 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:24.150447 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b523262-2aae-43c5-a358-6ec1624639d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:24.150729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b523262-2aae-43c5-a358-6ec1624639d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:24.151464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b523262-2aae-43c5-a358-6ec1624639d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:24.153323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b523262-2aae-43c5-a358-6ec1624639d1 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:24.165185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 900/1820] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:24.172712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe4b85-6b50-47b5-9981-b261e68cd1fd admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:24.182743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe4b85-6b50-47b5-9981-b261e68cd1fd admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:24.187391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596012f-fdcd-45df-b371-420dd0e97a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:24.188270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596012f-fdcd-45df-b371-420dd0e97a18 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:24.188555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596012f-fdcd-45df-b371-420dd0e97a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:24.188864 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596012f-fdcd-45df-b371-420dd0e97a18 admin admin] PATH_INFO: `/v3/projects/72a9e93c4b8045f2938433550fc7ce32/users/e614acbec9944ecfab50fcba33bb9b81/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:24.203325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596012f-fdcd-45df-b371-420dd0e97a18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a9e93c4b8045f2938433550fc7ce32, user_id=e614acbec9944ecfab50fcba33bb9b81, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:24.206032 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596012f-fdcd-45df-b371-420dd0e97a18 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:24.221096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 921/1821] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:24 2022] PUT /identity/v3/projects/72a9e93c4b8045f2938433550fc7ce32/users/e614acbec9944ecfab50fcba33bb9b81/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:24.224919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b523262-2aae-43c5-a358-6ec1624639d1 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:24.235573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b523262-2aae-43c5-a358-6ec1624639d1 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:24.291478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a380917-b03d-4198-b4a9-cae75b25fe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:24.292167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a380917-b03d-4198-b4a9-cae75b25fe96 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:24.292452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a380917-b03d-4198-b4a9-cae75b25fe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:24.292764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a380917-b03d-4198-b4a9-cae75b25fe96 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:24.293499 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a380917-b03d-4198-b4a9-cae75b25fe96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:24.295109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a380917-b03d-4198-b4a9-cae75b25fe96 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:24.306456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 901/1822] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:24.311110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596012f-fdcd-45df-b371-420dd0e97a18 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:24.321636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596012f-fdcd-45df-b371-420dd0e97a18 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:24.326317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba048f9f-76c1-4f04-ac7e-65f846ff275f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:24.327735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba048f9f-76c1-4f04-ac7e-65f846ff275f admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:24.328016 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba048f9f-76c1-4f04-ac7e-65f846ff275f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:24.328289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba048f9f-76c1-4f04-ac7e-65f846ff275f admin admin] PATH_INFO: `/v3/projects/72a9e93c4b8045f2938433550fc7ce32/users/e614acbec9944ecfab50fcba33bb9b81/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:24.330293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba048f9f-76c1-4f04-ac7e-65f846ff275f admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a9e93c4b8045f2938433550fc7ce32, user_id=e614acbec9944ecfab50fcba33bb9b81, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:24.332997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba048f9f-76c1-4f04-ac7e-65f846ff275f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:24.350095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 922/1823] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:24 2022] PUT /identity/v3/projects/72a9e93c4b8045f2938433550fc7ce32/users/e614acbec9944ecfab50fcba33bb9b81/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:24.361249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ad287-f0de-4e60-8933-12ebc322ed80 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:24.361573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ad287-f0de-4e60-8933-12ebc322ed80 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:24.361856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ad287-f0de-4e60-8933-12ebc322ed80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:24.386974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:24.390495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-d46ad287-f0de-4e60-8933-12ebc322ed80 None None] MFA Rules not processed for user `e614acbec9944ecfab50fcba33bb9b81`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:24.392306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d46ad287-f0de-4e60-8933-12ebc322ed80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:24.462327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 902/1824] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:24.474762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba048f9f-76c1-4f04-ac7e-65f846ff275f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:24.476615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ba048f9f-76c1-4f04-ac7e-65f846ff275f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:24.523532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:24.539999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba048f9f-76c1-4f04-ac7e-65f846ff275f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:24.543541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d73ba2-7ea0-412a-87bd-9eb75d651d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:24.544189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d73ba2-7ea0-412a-87bd-9eb75d651d83 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:24.544497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d73ba2-7ea0-412a-87bd-9eb75d651d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:24.544813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d73ba2-7ea0-412a-87bd-9eb75d651d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:24.554232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d73ba2-7ea0-412a-87bd-9eb75d651d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:24.555651 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d73ba2-7ea0-412a-87bd-9eb75d651d83 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:24.565837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 923/1825] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:24.629817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ad287-f0de-4e60-8933-12ebc322ed80 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:24.691535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ad287-f0de-4e60-8933-12ebc322ed80 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:24.696383 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:24.697377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:24.697660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:24.697935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] PATH_INFO: `/v3/users/21b700175b514febba6cada0301ef194` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:24.699163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21b700175b514febba6cada0301ef194)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:24.701105 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:24.748805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21b700175b514febba6cada0301ef194'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:24.757340 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21b700175b514febba6cada0301ef194'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:24.770460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21b700175b514febba6cada0301ef194'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:24.778990 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21b700175b514febba6cada0301ef194'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:24.779679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21b700175b514febba6cada0301ef194'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:24.786618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 903/1826] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:24 2022] DELETE /identity/v3/users/21b700175b514febba6cada0301ef194 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:24.790858 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58ab8a-bed0-4d71-9097-3885cfa91885 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:24.791083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58ab8a-bed0-4d71-9097-3885cfa91885 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:24.791366 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58ab8a-bed0-4d71-9097-3885cfa91885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:24.811309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4c58ab8a-bed0-4d71-9097-3885cfa91885 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:24.812564 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4c58ab8a-bed0-4d71-9097-3885cfa91885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:24.878373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 924/1827] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:24.888133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:24.889593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:24.910087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:24.950454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7dcd6-a861-4c76-b42a-ff3e11d2abc6 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:24.955012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cc3f8-590b-47ca-b4a9-19482207a185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:24.955678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cc3f8-590b-47ca-b4a9-19482207a185 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:24.955965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cc3f8-590b-47ca-b4a9-19482207a185 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:24.956243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cc3f8-590b-47ca-b4a9-19482207a185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:24.965875 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cc3f8-590b-47ca-b4a9-19482207a185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:24.967305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cc3f8-590b-47ca-b4a9-19482207a185 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:24.977777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 904/1828] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:25.274143 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58ab8a-bed0-4d71-9097-3885cfa91885 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:25.275689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4c58ab8a-bed0-4d71-9097-3885cfa91885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:25.288033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58ab8a-bed0-4d71-9097-3885cfa91885 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:25.292789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:25.293679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:25.293939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:25.294220 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] PATH_INFO: `/v3/projects/8e6cb16388ca400e8f9e193a6b98df11` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:25.295433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e6cb16388ca400e8f9e193a6b98df11)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:25.297464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:25.361028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e6cb16388ca400e8f9e193a6b98df11', 'request_id': 'req-e9fa4bae-abc3-4d90-977e-351314045982'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:25.370188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e6cb16388ca400e8f9e193a6b98df11', 'request_id': 'req-e9fa4bae-abc3-4d90-977e-351314045982'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:25.376411 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e6cb16388ca400e8f9e193a6b98df11', 'request_id': 'req-e9fa4bae-abc3-4d90-977e-351314045982'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:25.388371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] The token cache is being invalidate because project 8e6cb16388ca400e8f9e193a6b98df11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:25.388824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:25.390088 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:25.391657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 925/1829] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:25 2022] DELETE /identity/v3/projects/8e6cb16388ca400e8f9e193a6b98df11 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:25.395580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cc3f8-590b-47ca-b4a9-19482207a185 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:25.405930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cc3f8-590b-47ca-b4a9-19482207a185 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:25.462283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea25202-62ec-4230-be4c-b79437d25008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:25.463165 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea25202-62ec-4230-be4c-b79437d25008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:25.463451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea25202-62ec-4230-be4c-b79437d25008 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:25.463730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea25202-62ec-4230-be4c-b79437d25008 admin admin] PATH_INFO: `/v3/users/e614acbec9944ecfab50fcba33bb9b81` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:25.464957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea25202-62ec-4230-be4c-b79437d25008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e614acbec9944ecfab50fcba33bb9b81)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:25.466922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea25202-62ec-4230-be4c-b79437d25008 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:25.511831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-6ea25202-62ec-4230-be4c-b79437d25008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e614acbec9944ecfab50fcba33bb9b81'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:25.520252 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-6ea25202-62ec-4230-be4c-b79437d25008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e614acbec9944ecfab50fcba33bb9b81'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:25.534740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-6ea25202-62ec-4230-be4c-b79437d25008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e614acbec9944ecfab50fcba33bb9b81'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:25.542338 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-6ea25202-62ec-4230-be4c-b79437d25008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e614acbec9944ecfab50fcba33bb9b81'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:25.543075 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-6ea25202-62ec-4230-be4c-b79437d25008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e614acbec9944ecfab50fcba33bb9b81'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:25.549729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 905/1830] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:25 2022] DELETE /identity/v3/users/e614acbec9944ecfab50fcba33bb9b81 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:25.849409 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:25.850962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:25.871466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa4bae-abc3-4d90-977e-351314045982 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:25.945076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763e893-a11b-4694-a8e5-f422cbe437a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:25.945952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763e893-a11b-4694-a8e5-f422cbe437a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:25.946234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763e893-a11b-4694-a8e5-f422cbe437a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:25.946512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763e893-a11b-4694-a8e5-f422cbe437a8 admin admin] PATH_INFO: `/v3/projects/72a9e93c4b8045f2938433550fc7ce32` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:25.947770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763e893-a11b-4694-a8e5-f422cbe437a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72a9e93c4b8045f2938433550fc7ce32)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:25.949724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763e893-a11b-4694-a8e5-f422cbe437a8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:25.957429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:26.001240 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f763e893-a11b-4694-a8e5-f422cbe437a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72a9e93c4b8045f2938433550fc7ce32', 'request_id': 'req-f763e893-a11b-4694-a8e5-f422cbe437a8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:26.009216 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f763e893-a11b-4694-a8e5-f422cbe437a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72a9e93c4b8045f2938433550fc7ce32', 'request_id': 'req-f763e893-a11b-4694-a8e5-f422cbe437a8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:26.016105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f763e893-a11b-4694-a8e5-f422cbe437a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72a9e93c4b8045f2938433550fc7ce32', 'request_id': 'req-f763e893-a11b-4694-a8e5-f422cbe437a8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:26.027773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f763e893-a11b-4694-a8e5-f422cbe437a8 admin admin] The token cache is being invalidate because project 72a9e93c4b8045f2938433550fc7ce32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:26.028065 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f763e893-a11b-4694-a8e5-f422cbe437a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:26.028809 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f763e893-a11b-4694-a8e5-f422cbe437a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:26.031129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 926/1831] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:25 2022] DELETE /identity/v3/projects/72a9e93c4b8045f2938433550fc7ce32 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:26.036140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c795f7f-1fb1-428f-8e47-dd90195bf7e0 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.036431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c795f7f-1fb1-428f-8e47-dd90195bf7e0 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.036747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c795f7f-1fb1-428f-8e47-dd90195bf7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.060624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5c795f7f-1fb1-428f-8e47-dd90195bf7e0 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:26.061858 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5c795f7f-1fb1-428f-8e47-dd90195bf7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:26.095007 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:26.127785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 906/1832] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:26.135935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1476687-d001-4afa-b5e5-4444939484d8 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.136279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1476687-d001-4afa-b5e5-4444939484d8 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.136468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1476687-d001-4afa-b5e5-4444939484d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.157851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-f1476687-d001-4afa-b5e5-4444939484d8 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:26.159074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f1476687-d001-4afa-b5e5-4444939484d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:26.167104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 927/1833] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:26.171654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c795f7f-1fb1-428f-8e47-dd90195bf7e0 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:26.181214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c795f7f-1fb1-428f-8e47-dd90195bf7e0 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:26.185792 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cc96e-031a-4913-a69c-b836cbb0b135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:26.186988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc96e-031a-4913-a69c-b836cbb0b135 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.187270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc96e-031a-4913-a69c-b836cbb0b135 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.187549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc96e-031a-4913-a69c-b836cbb0b135 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.188277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cc96e-031a-4913-a69c-b836cbb0b135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:26.188602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cc96e-031a-4913-a69c-b836cbb0b135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:26.190231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cc96e-031a-4913-a69c-b836cbb0b135 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:26.201919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 907/1834] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:46:26.206866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1476687-d001-4afa-b5e5-4444939484d8 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:26.216034 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1476687-d001-4afa-b5e5-4444939484d8 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:26.220970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c995e-8e22-484c-9dc4-bc488a98a34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:26.221845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c995e-8e22-484c-9dc4-bc488a98a34b admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.222134 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c995e-8e22-484c-9dc4-bc488a98a34b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.222424 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c995e-8e22-484c-9dc4-bc488a98a34b admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.223138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c995e-8e22-484c-9dc4-bc488a98a34b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:26.225112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c995e-8e22-484c-9dc4-bc488a98a34b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:26.255096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 928/1835] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:26 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:26.261922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cc96e-031a-4913-a69c-b836cbb0b135 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:26.272331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cc96e-031a-4913-a69c-b836cbb0b135 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:26.329213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1e6f4-e2bc-4963-8571-938be5e03125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:26.329921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1e6f4-e2bc-4963-8571-938be5e03125 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.330202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1e6f4-e2bc-4963-8571-938be5e03125 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.330478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1e6f4-e2bc-4963-8571-938be5e03125 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.331174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1e6f4-e2bc-4963-8571-938be5e03125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:26.333120 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1e6f4-e2bc-4963-8571-938be5e03125 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:26.355284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 908/1836] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:26 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:26.359905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c995e-8e22-484c-9dc4-bc488a98a34b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:26.370393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c995e-8e22-484c-9dc4-bc488a98a34b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:26.375140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203cd7f-5bb1-4937-b3ae-dd7cd0beb78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:26.375814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203cd7f-5bb1-4937-b3ae-dd7cd0beb78f admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.376229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203cd7f-5bb1-4937-b3ae-dd7cd0beb78f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.376433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203cd7f-5bb1-4937-b3ae-dd7cd0beb78f admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.377171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203cd7f-5bb1-4937-b3ae-dd7cd0beb78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:26.379033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203cd7f-5bb1-4937-b3ae-dd7cd0beb78f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:26.390387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 929/1837] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:26.394600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1e6f4-e2bc-4963-8571-938be5e03125 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:26.404885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1e6f4-e2bc-4963-8571-938be5e03125 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:26.410283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c040a-eea3-4a7c-86da-baec3e8b199e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:26.410973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c040a-eea3-4a7c-86da-baec3e8b199e admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.411262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c040a-eea3-4a7c-86da-baec3e8b199e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.411538 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c040a-eea3-4a7c-86da-baec3e8b199e admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.412271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c040a-eea3-4a7c-86da-baec3e8b199e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:26.414252 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c040a-eea3-4a7c-86da-baec3e8b199e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:26.425893 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 909/1838] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:26.430228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203cd7f-5bb1-4937-b3ae-dd7cd0beb78f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:26.440587 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203cd7f-5bb1-4937-b3ae-dd7cd0beb78f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:26.447254 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a684f-6f83-4aba-ac4b-ebf1e4d44f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:26.448959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a684f-6f83-4aba-ac4b-ebf1e4d44f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.449242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a684f-6f83-4aba-ac4b-ebf1e4d44f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.449518 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a684f-6f83-4aba-ac4b-ebf1e4d44f90 admin admin] PATH_INFO: `/v3/projects/fc59715466444a12989701ac41a571ae/users/9421a10485164699baf904cb94b8a20c/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.468240 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a684f-6f83-4aba-ac4b-ebf1e4d44f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc59715466444a12989701ac41a571ae, user_id=9421a10485164699baf904cb94b8a20c, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:26.471577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a684f-6f83-4aba-ac4b-ebf1e4d44f90 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:26.489836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 930/1839] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:26 2022] PUT /identity/v3/projects/fc59715466444a12989701ac41a571ae/users/9421a10485164699baf904cb94b8a20c/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:26.500345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff411bd-a8e4-4555-936e-dd5a27079c6e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.500768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff411bd-a8e4-4555-936e-dd5a27079c6e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.501011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff411bd-a8e4-4555-936e-dd5a27079c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.536248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-cff411bd-a8e4-4555-936e-dd5a27079c6e None None] MFA Rules not processed for user `9421a10485164699baf904cb94b8a20c`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:26.537586 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-cff411bd-a8e4-4555-936e-dd5a27079c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:26.603097 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 910/1840] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:26.607700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a684f-6f83-4aba-ac4b-ebf1e4d44f90 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:26.680138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a684f-6f83-4aba-ac4b-ebf1e4d44f90 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:26.688749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19e5e7-9a9d-409b-aae6-7c87be4c7f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:26.689500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19e5e7-9a9d-409b-aae6-7c87be4c7f2d admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.689791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19e5e7-9a9d-409b-aae6-7c87be4c7f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.690070 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19e5e7-9a9d-409b-aae6-7c87be4c7f2d admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.690786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19e5e7-9a9d-409b-aae6-7c87be4c7f2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:26.692893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19e5e7-9a9d-409b-aae6-7c87be4c7f2d admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:26.730003 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 931/1841] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:26 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:26.734257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff411bd-a8e4-4555-936e-dd5a27079c6e None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:26.747719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff411bd-a8e4-4555-936e-dd5a27079c6e None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:26.783522 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:26.812202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e2b02-0816-49e3-8b99-4579669c2ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:26.812921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2b02-0816-49e3-8b99-4579669c2ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.813197 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2b02-0816-49e3-8b99-4579669c2ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.813478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2b02-0816-49e3-8b99-4579669c2ca2 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.814190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e2b02-0816-49e3-8b99-4579669c2ca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:26.816129 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e2b02-0816-49e3-8b99-4579669c2ca2 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:26.841522 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 911/1842] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:26 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:26.842815 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19e5e7-9a9d-409b-aae6-7c87be4c7f2d admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:26.853311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19e5e7-9a9d-409b-aae6-7c87be4c7f2d admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:26.857985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716eb1b-ff97-4c6f-89be-2984e5a771a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:26.859186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716eb1b-ff97-4c6f-89be-2984e5a771a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.859469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716eb1b-ff97-4c6f-89be-2984e5a771a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.859978 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716eb1b-ff97-4c6f-89be-2984e5a771a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.860787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716eb1b-ff97-4c6f-89be-2984e5a771a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:26.862393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716eb1b-ff97-4c6f-89be-2984e5a771a2 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:26.878825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 932/1843] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:26.882765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e2b02-0816-49e3-8b99-4579669c2ca2 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:26.893112 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e2b02-0816-49e3-8b99-4579669c2ca2 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:26.899518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0beb8b-bd4c-4888-8a6d-491309e3d307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:26.900867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0beb8b-bd4c-4888-8a6d-491309e3d307 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.901155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0beb8b-bd4c-4888-8a6d-491309e3d307 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.901434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0beb8b-bd4c-4888-8a6d-491309e3d307 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.902174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0beb8b-bd4c-4888-8a6d-491309e3d307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:26.903777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0beb8b-bd4c-4888-8a6d-491309e3d307 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:26.914697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 912/1844] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:26.919053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716eb1b-ff97-4c6f-89be-2984e5a771a2 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:26.927161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:26.936029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716eb1b-ff97-4c6f-89be-2984e5a771a2 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:26.938706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6fd1c-2d97-4e4b-85bc-e3a274b1d047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:26.939884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6fd1c-2d97-4e4b-85bc-e3a274b1d047 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.942133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6fd1c-2d97-4e4b-85bc-e3a274b1d047 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.942415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6fd1c-2d97-4e4b-85bc-e3a274b1d047 admin admin] PATH_INFO: `/v3/projects/7cbe3002e3da4cbea85c7a0c02f9f440/users/51727a0d38cd4721ba4333763b65c47a/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:26.966418 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6fd1c-2d97-4e4b-85bc-e3a274b1d047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbe3002e3da4cbea85c7a0c02f9f440, user_id=51727a0d38cd4721ba4333763b65c47a, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:26.969644 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6fd1c-2d97-4e4b-85bc-e3a274b1d047 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:26.981898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 933/1845] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:26 2022] PUT /identity/v3/projects/7cbe3002e3da4cbea85c7a0c02f9f440/users/51727a0d38cd4721ba4333763b65c47a/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:26.993733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250984ab-fae6-49d2-9e40-1a3704f49d41 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:26.994033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250984ab-fae6-49d2-9e40-1a3704f49d41 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:26.994317 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250984ab-fae6-49d2-9e40-1a3704f49d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:27.022575 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-250984ab-fae6-49d2-9e40-1a3704f49d41 None None] MFA Rules not processed for user `51727a0d38cd4721ba4333763b65c47a`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:27.023922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-250984ab-fae6-49d2-9e40-1a3704f49d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:27.090378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 913/1846] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:27.095724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6fd1c-2d97-4e4b-85bc-e3a274b1d047 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:27.163128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6fd1c-2d97-4e4b-85bc-e3a274b1d047 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:27.167820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683446f-7e7e-436c-80b5-3a9416d64005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:27.168493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683446f-7e7e-436c-80b5-3a9416d64005 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:27.168812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683446f-7e7e-436c-80b5-3a9416d64005 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:27.169095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683446f-7e7e-436c-80b5-3a9416d64005 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:27.169814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683446f-7e7e-436c-80b5-3a9416d64005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:27.171813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683446f-7e7e-436c-80b5-3a9416d64005 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:27.200602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 934/1847] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:27 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:27.204493 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250984ab-fae6-49d2-9e40-1a3704f49d41 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:27.215347 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250984ab-fae6-49d2-9e40-1a3704f49d41 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:27.270935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eac156-bd7c-4ef3-8cfc-95cc48dfa318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:27.271618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eac156-bd7c-4ef3-8cfc-95cc48dfa318 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:27.271903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eac156-bd7c-4ef3-8cfc-95cc48dfa318 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:27.272183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eac156-bd7c-4ef3-8cfc-95cc48dfa318 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:27.272938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eac156-bd7c-4ef3-8cfc-95cc48dfa318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:27.274869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eac156-bd7c-4ef3-8cfc-95cc48dfa318 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:27.298815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 914/1848] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:27 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:27.302752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683446f-7e7e-436c-80b5-3a9416d64005 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:27.313687 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683446f-7e7e-436c-80b5-3a9416d64005 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:27.318396 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add77bdf-9672-4515-b936-d8a498c44999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:27.319079 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add77bdf-9672-4515-b936-d8a498c44999 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:27.319369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add77bdf-9672-4515-b936-d8a498c44999 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:27.319653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add77bdf-9672-4515-b936-d8a498c44999 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:27.320396 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add77bdf-9672-4515-b936-d8a498c44999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:27.322373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add77bdf-9672-4515-b936-d8a498c44999 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:27.334454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 935/1849] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:27.338700 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eac156-bd7c-4ef3-8cfc-95cc48dfa318 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:27.349234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eac156-bd7c-4ef3-8cfc-95cc48dfa318 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:27.354162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8758836-e10c-4c52-a391-07456c6c8652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:27.355091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8758836-e10c-4c52-a391-07456c6c8652 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:27.355394 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8758836-e10c-4c52-a391-07456c6c8652 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:27.355656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8758836-e10c-4c52-a391-07456c6c8652 admin admin] PATH_INFO: `/v3/projects/8996257b8c53407491875eb792ae3b19/users/32597b3dc97e46259fa34ce6eefebe0a/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:27.371074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8758836-e10c-4c52-a391-07456c6c8652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8996257b8c53407491875eb792ae3b19, user_id=32597b3dc97e46259fa34ce6eefebe0a, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:27.373791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8758836-e10c-4c52-a391-07456c6c8652 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:27.388117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 915/1850] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:27 2022] PUT /identity/v3/projects/8996257b8c53407491875eb792ae3b19/users/32597b3dc97e46259fa34ce6eefebe0a/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:27.392556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add77bdf-9672-4515-b936-d8a498c44999 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:27.403682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add77bdf-9672-4515-b936-d8a498c44999 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:27.464023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950739cb-6b57-490d-9bd3-02662431aa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:27.464729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950739cb-6b57-490d-9bd3-02662431aa7a admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:27.465019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950739cb-6b57-490d-9bd3-02662431aa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:27.465304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950739cb-6b57-490d-9bd3-02662431aa7a admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:27.466033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950739cb-6b57-490d-9bd3-02662431aa7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:27.467674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950739cb-6b57-490d-9bd3-02662431aa7a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:27.477817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 936/1851] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:27.481974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8758836-e10c-4c52-a391-07456c6c8652 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:27.492090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8758836-e10c-4c52-a391-07456c6c8652 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:27.496619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee135a-0ff4-4592-87a7-38545c11fbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:27.497530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee135a-0ff4-4592-87a7-38545c11fbf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:27.497815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee135a-0ff4-4592-87a7-38545c11fbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:27.498086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee135a-0ff4-4592-87a7-38545c11fbf5 admin admin] PATH_INFO: `/v3/projects/8996257b8c53407491875eb792ae3b19/users/32597b3dc97e46259fa34ce6eefebe0a/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:27.500067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee135a-0ff4-4592-87a7-38545c11fbf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8996257b8c53407491875eb792ae3b19, user_id=32597b3dc97e46259fa34ce6eefebe0a, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:27.502707 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee135a-0ff4-4592-87a7-38545c11fbf5 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:27.505678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:27.515031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 916/1852] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:27 2022] PUT /identity/v3/projects/8996257b8c53407491875eb792ae3b19/users/32597b3dc97e46259fa34ce6eefebe0a/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:27.525203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8b852-3233-4035-81bb-5b4b1dde1fea None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:27.525495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8b852-3233-4035-81bb-5b4b1dde1fea None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:27.525773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8b852-3233-4035-81bb-5b4b1dde1fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:27.556060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-aad8b852-3233-4035-81bb-5b4b1dde1fea None None] MFA Rules not processed for user `32597b3dc97e46259fa34ce6eefebe0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:27.557346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-aad8b852-3233-4035-81bb-5b4b1dde1fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:27.605595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:27.637461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 937/1853] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:27.649181 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee135a-0ff4-4592-87a7-38545c11fbf5 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:27.651027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5dee135a-0ff4-4592-87a7-38545c11fbf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:27.711704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee135a-0ff4-4592-87a7-38545c11fbf5 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:27.715345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3664da3-46c9-4707-aa66-daeb862ca517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:27.715999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3664da3-46c9-4707-aa66-daeb862ca517 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:27.716281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3664da3-46c9-4707-aa66-daeb862ca517 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:27.716558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3664da3-46c9-4707-aa66-daeb862ca517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:27.725659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3664da3-46c9-4707-aa66-daeb862ca517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:27.727099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3664da3-46c9-4707-aa66-daeb862ca517 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:27.736861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 917/1854] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:27.779698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8b852-3233-4035-81bb-5b4b1dde1fea None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:27.781157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-aad8b852-3233-4035-81bb-5b4b1dde1fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:27.793611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8b852-3233-4035-81bb-5b4b1dde1fea None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:27.797263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6a88c-c965-446d-89d4-edcff342a4b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:27.797915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6a88c-c965-446d-89d4-edcff342a4b3 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:27.798200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6a88c-c965-446d-89d4-edcff342a4b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:27.798480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6a88c-c965-446d-89d4-edcff342a4b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:27.859582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6a88c-c965-446d-89d4-edcff342a4b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:27.861072 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6a88c-c965-446d-89d4-edcff342a4b3 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:27.869880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 938/1855] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:35.349141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3664da3-46c9-4707-aa66-daeb862ca517 service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:35.351153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b3664da3-46c9-4707-aa66-daeb862ca517 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:35.431510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3664da3-46c9-4707-aa66-daeb862ca517 service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:35.436797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:35.437695 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:35.437985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:35.438266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] PATH_INFO: `/v3/users/8b7d9d692bc84388b52bf74ef8011a74` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:35.439517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7d9d692bc84388b52bf74ef8011a74)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:35.441475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:35.496896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7d9d692bc84388b52bf74ef8011a74'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:35.507371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7d9d692bc84388b52bf74ef8011a74'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:35.525563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7d9d692bc84388b52bf74ef8011a74'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:35.535271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7d9d692bc84388b52bf74ef8011a74'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:35.536720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7d9d692bc84388b52bf74ef8011a74'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:35.543509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 918/1856] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:35 2022] DELETE /identity/v3/users/8b7d9d692bc84388b52bf74ef8011a74 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:35.549331 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594ed74-8caf-4bf3-92de-bf0f4986694a None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:35.550217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594ed74-8caf-4bf3-92de-bf0f4986694a None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:35.550676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594ed74-8caf-4bf3-92de-bf0f4986694a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:35.576393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-f594ed74-8caf-4bf3-92de-bf0f4986694a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:35.577742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f594ed74-8caf-4bf3-92de-bf0f4986694a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:35.668204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 939/1857] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:35.676520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:35.680364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:35.709915 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:35.789986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163b049-a310-4c6b-af16-06f36a539545 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:35.801769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20a392-e560-44df-890e-e45740bc4e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:35.802952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20a392-e560-44df-890e-e45740bc4e1d service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:35.803351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20a392-e560-44df-890e-e45740bc4e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:35.803738 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20a392-e560-44df-890e-e45740bc4e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:35.816402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20a392-e560-44df-890e-e45740bc4e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:35.818145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20a392-e560-44df-890e-e45740bc4e1d service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:35.830041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 919/1858] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:36.168675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594ed74-8caf-4bf3-92de-bf0f4986694a None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:36.170191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f594ed74-8caf-4bf3-92de-bf0f4986694a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:36.183050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594ed74-8caf-4bf3-92de-bf0f4986694a None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:36.187804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:36.188718 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:36.189005 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:36.189285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] PATH_INFO: `/v3/users/9421a10485164699baf904cb94b8a20c` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:36.190675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9421a10485164699baf904cb94b8a20c)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:36.193051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:36.243039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9421a10485164699baf904cb94b8a20c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.243698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9421a10485164699baf904cb94b8a20c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.253502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9421a10485164699baf904cb94b8a20c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.266093 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:36.268699 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9421a10485164699baf904cb94b8a20c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.281675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9421a10485164699baf904cb94b8a20c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.289682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 940/1859] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:36 2022] DELETE /identity/v3/users/9421a10485164699baf904cb94b8a20c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:36.294258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ccaa1-26ca-451d-837f-664bc345bfd7 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:36.294492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ccaa1-26ca-451d-837f-664bc345bfd7 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:36.294781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ccaa1-26ca-451d-837f-664bc345bfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:36.307770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:36.310251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:36.319165 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-373ccaa1-26ca-451d-837f-664bc345bfd7 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:36.320387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-373ccaa1-26ca-451d-837f-664bc345bfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:36.380528 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c1ba3-11fc-43c6-9d34-ca6c52340986 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:36.385367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:36.385754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 920/1860] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:36.386275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:36.386535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:36.386790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] PATH_INFO: `/v3/projects/2f624edca273431798ca934aef196908` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:36.387972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f624edca273431798ca934aef196908)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:36.390437 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:36.395917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ccaa1-26ca-451d-837f-664bc345bfd7 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:36.397362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-373ccaa1-26ca-451d-837f-664bc345bfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:36.439151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f624edca273431798ca934aef196908', 'request_id': 'req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.446496 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f624edca273431798ca934aef196908', 'request_id': 'req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.452945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f624edca273431798ca934aef196908', 'request_id': 'req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.456831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ccaa1-26ca-451d-837f-664bc345bfd7 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:36.464111 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] The token cache is being invalidate because project 2f624edca273431798ca934aef196908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:36.464405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.465078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.466654 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 941/1861] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:36 2022] DELETE /identity/v3/projects/2f624edca273431798ca934aef196908 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:36.471054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:36.472361 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:36.505962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01e5a0-d703-4462-8915-9bcd6a59b790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:36.506618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01e5a0-d703-4462-8915-9bcd6a59b790 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:36.506900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01e5a0-d703-4462-8915-9bcd6a59b790 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:36.507176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01e5a0-d703-4462-8915-9bcd6a59b790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:36.508950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2c01e5a0-d703-4462-8915-9bcd6a59b790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:36.536660 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e680ae-4f45-4d21-bcb0-1e9006bda8d7 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:36.541930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5532a-41fb-4fd2-a08c-d60e442035c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:36.542804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5532a-41fb-4fd2-a08c-d60e442035c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:36.543089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5532a-41fb-4fd2-a08c-d60e442035c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:36.543557 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5532a-41fb-4fd2-a08c-d60e442035c8 admin admin] PATH_INFO: `/v3/users/43fdc49753e64a05889bb7da8ab76235` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:36.544877 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5532a-41fb-4fd2-a08c-d60e442035c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43fdc49753e64a05889bb7da8ab76235)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:36.547412 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:36.547663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5532a-41fb-4fd2-a08c-d60e442035c8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:36.574530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:36.594937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01e5a0-d703-4462-8915-9bcd6a59b790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:36.596673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01e5a0-d703-4462-8915-9bcd6a59b790 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:36.609702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 921/1862] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:46:36.610728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2bf5532a-41fb-4fd2-a08c-d60e442035c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43fdc49753e64a05889bb7da8ab76235'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.611464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2bf5532a-41fb-4fd2-a08c-d60e442035c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43fdc49753e64a05889bb7da8ab76235'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.621720 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2bf5532a-41fb-4fd2-a08c-d60e442035c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43fdc49753e64a05889bb7da8ab76235'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.639782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2bf5532a-41fb-4fd2-a08c-d60e442035c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43fdc49753e64a05889bb7da8ab76235'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.650279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2bf5532a-41fb-4fd2-a08c-d60e442035c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43fdc49753e64a05889bb7da8ab76235'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:36.662000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 942/1863] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:36 2022] DELETE /identity/v3/users/43fdc49753e64a05889bb7da8ab76235 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:36.993404 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2958bcf-972d-4e4a-a37c-7caaa533a388 None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:36.993694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2958bcf-972d-4e4a-a37c-7caaa533a388 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:36.995957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2958bcf-972d-4e4a-a37c-7caaa533a388 None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:36.995957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 922/1864] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:46:37.026289 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2958bcf-972d-4e4a-a37c-7caaa533a388 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:37.028887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d801ce8-9fa5-4db4-b82d-7eab6cfb2dd3 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:37.029468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d801ce8-9fa5-4db4-b82d-7eab6cfb2dd3 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:37.032105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d801ce8-9fa5-4db4-b82d-7eab6cfb2dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:37.032632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c2958bcf-972d-4e4a-a37c-7caaa533a388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:37.081361 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-8d801ce8-9fa5-4db4-b82d-7eab6cfb2dd3 None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:37.082730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8d801ce8-9fa5-4db4-b82d-7eab6cfb2dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:37.254020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2958bcf-972d-4e4a-a37c-7caaa533a388 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:37.261282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:37.262346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:37.262766 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:37.263186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] PATH_INFO: `/v3/projects/fc59715466444a12989701ac41a571ae` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:37.265584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc59715466444a12989701ac41a571ae)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:37.267133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 943/1865] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:46:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:37.268444 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:37.269495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d801ce8-9fa5-4db4-b82d-7eab6cfb2dd3 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:37.270926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8d801ce8-9fa5-4db4-b82d-7eab6cfb2dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:37.287876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d801ce8-9fa5-4db4-b82d-7eab6cfb2dd3 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:37.294834 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:37.296299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:37.296702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:37.296982 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] PATH_INFO: `/v3/projects/91e54e396e2c48ed9b407c2966a6a84b` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:37.298249 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e54e396e2c48ed9b407c2966a6a84b)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:37.300527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:37.363467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc59715466444a12989701ac41a571ae', 'request_id': 'req-06d2585f-ca2d-4f14-a363-2703d6c5dac5'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.377384 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc59715466444a12989701ac41a571ae', 'request_id': 'req-06d2585f-ca2d-4f14-a363-2703d6c5dac5'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.379241 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e54e396e2c48ed9b407c2966a6a84b', 'request_id': 'req-66722a1d-7c39-49ce-9158-03e04af23038'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.388166 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc59715466444a12989701ac41a571ae', 'request_id': 'req-06d2585f-ca2d-4f14-a363-2703d6c5dac5'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.393537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e54e396e2c48ed9b407c2966a6a84b', 'request_id': 'req-66722a1d-7c39-49ce-9158-03e04af23038'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.407094 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e54e396e2c48ed9b407c2966a6a84b', 'request_id': 'req-66722a1d-7c39-49ce-9158-03e04af23038'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.407966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] The token cache is being invalidate because project fc59715466444a12989701ac41a571ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:37.408731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.409912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.416611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 923/1866] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:37 2022] DELETE /identity/v3/projects/fc59715466444a12989701ac41a571ae => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:37.417899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:37.419539 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:37.424886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] The token cache is being invalidate because project 91e54e396e2c48ed9b407c2966a6a84b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:37.425361 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.426039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.428364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 944/1867] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:37 2022] DELETE /identity/v3/projects/91e54e396e2c48ed9b407c2966a6a84b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:37.429196 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:37.433357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:37.460523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:37.509186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:37.539186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66722a1d-7c39-49ce-9158-03e04af23038 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:37.544508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae43c2-6d27-4d92-8a2a-19462b652de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:37.545680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae43c2-6d27-4d92-8a2a-19462b652de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:37.545945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae43c2-6d27-4d92-8a2a-19462b652de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:37.546222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae43c2-6d27-4d92-8a2a-19462b652de9 admin admin] PATH_INFO: `/v3/users/51727a0d38cd4721ba4333763b65c47a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:37.547654 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae43c2-6d27-4d92-8a2a-19462b652de9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51727a0d38cd4721ba4333763b65c47a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:37.550112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae43c2-6d27-4d92-8a2a-19462b652de9 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:37.563132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2585f-ca2d-4f14-a363-2703d6c5dac5 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:37.662197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-25ae43c2-6d27-4d92-8a2a-19462b652de9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51727a0d38cd4721ba4333763b65c47a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.663641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-25ae43c2-6d27-4d92-8a2a-19462b652de9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51727a0d38cd4721ba4333763b65c47a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.683614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-25ae43c2-6d27-4d92-8a2a-19462b652de9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51727a0d38cd4721ba4333763b65c47a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.709446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58607f92-c6ab-44f2-9512-9f595f3d45c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:37.711384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-25ae43c2-6d27-4d92-8a2a-19462b652de9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51727a0d38cd4721ba4333763b65c47a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.711782 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58607f92-c6ab-44f2-9512-9f595f3d45c9 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:37.712789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58607f92-c6ab-44f2-9512-9f595f3d45c9 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:37.713169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58607f92-c6ab-44f2-9512-9f595f3d45c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:37.717884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-58607f92-c6ab-44f2-9512-9f595f3d45c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:37.738526 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-25ae43c2-6d27-4d92-8a2a-19462b652de9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51727a0d38cd4721ba4333763b65c47a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:37.754975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 945/1868] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:37 2022] DELETE /identity/v3/users/51727a0d38cd4721ba4333763b65c47a => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:37.756934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a8f00-877c-4e28-8b16-a40ef75bc21f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:37.759016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58607f92-c6ab-44f2-9512-9f595f3d45c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:37.765856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a8f00-877c-4e28-8b16-a40ef75bc21f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:37.766155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a8f00-877c-4e28-8b16-a40ef75bc21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:37.767056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58607f92-c6ab-44f2-9512-9f595f3d45c9 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:37.792266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 924/1869] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:37.809469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a16a8f00-877c-4e28-8b16-a40ef75bc21f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:37.810712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a16a8f00-877c-4e28-8b16-a40ef75bc21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:37.881444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 946/1870] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:37.889576 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c46aea-edf1-4cd5-9f2a-b50f080e9069 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:37.889868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c46aea-edf1-4cd5-9f2a-b50f080e9069 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:37.890151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c46aea-edf1-4cd5-9f2a-b50f080e9069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:37.913331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-14c46aea-edf1-4cd5-9f2a-b50f080e9069 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:37.914566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-14c46aea-edf1-4cd5-9f2a-b50f080e9069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:37.927581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 925/1871] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:37.928789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a8f00-877c-4e28-8b16-a40ef75bc21f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:37.939510 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a8f00-877c-4e28-8b16-a40ef75bc21f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:37.952330 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9d881-f34c-4e5d-a033-f8e22998f6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:37.953064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9d881-f34c-4e5d-a033-f8e22998f6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:37.953433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9d881-f34c-4e5d-a033-f8e22998f6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:37.953812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9d881-f34c-4e5d-a033-f8e22998f6b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:37.954469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9d881-f34c-4e5d-a033-f8e22998f6b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:37.954778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9d881-f34c-4e5d-a033-f8e22998f6b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:37.960454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9d881-f34c-4e5d-a033-f8e22998f6b9 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:37.977634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 947/1872] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:46:37.978759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c46aea-edf1-4cd5-9f2a-b50f080e9069 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:37.988505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c46aea-edf1-4cd5-9f2a-b50f080e9069 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:37.993180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20ab2d-9f69-4e27-86c0-6b1898e52298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:37.993831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20ab2d-9f69-4e27-86c0-6b1898e52298 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:37.994113 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20ab2d-9f69-4e27-86c0-6b1898e52298 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:37.994390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20ab2d-9f69-4e27-86c0-6b1898e52298 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:37.995084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20ab2d-9f69-4e27-86c0-6b1898e52298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:37.997013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20ab2d-9f69-4e27-86c0-6b1898e52298 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.024701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 926/1873] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:37 2022] POST /identity/v3/projects => generated 401 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:38.029394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9d881-f34c-4e5d-a033-f8e22998f6b9 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:38.040049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20ab2d-9f69-4e27-86c0-6b1898e52298 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:38.041840 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9d881-f34c-4e5d-a033-f8e22998f6b9 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:38.042023 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ae20ab2d-9f69-4e27-86c0-6b1898e52298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:38.061984 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:38.110590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b665b-8812-4dc7-9b86-ae123cc02b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:38.111277 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b665b-8812-4dc7-9b86-ae123cc02b53 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.111566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b665b-8812-4dc7-9b86-ae123cc02b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.111847 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b665b-8812-4dc7-9b86-ae123cc02b53 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.112547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b665b-8812-4dc7-9b86-ae123cc02b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:38.113171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20ab2d-9f69-4e27-86c0-6b1898e52298 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:38.114502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b665b-8812-4dc7-9b86-ae123cc02b53 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.117884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c860b-af74-465e-a49a-941047582cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:38.118816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c860b-af74-465e-a49a-941047582cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.119171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c860b-af74-465e-a49a-941047582cae admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.119515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c860b-af74-465e-a49a-941047582cae admin admin] PATH_INFO: `/v3/projects/7cbe3002e3da4cbea85c7a0c02f9f440` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.120802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c860b-af74-465e-a49a-941047582cae admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cbe3002e3da4cbea85c7a0c02f9f440)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:38.122791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c860b-af74-465e-a49a-941047582cae admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.139919 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 948/1874] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:38 2022] POST /identity/v3/users => generated 301 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:38.141889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b665b-8812-4dc7-9b86-ae123cc02b53 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:38.153080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b665b-8812-4dc7-9b86-ae123cc02b53 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:38.158678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43328cd-700b-4b74-9d22-57adfcb12cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:38.159371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43328cd-700b-4b74-9d22-57adfcb12cfc admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.159659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43328cd-700b-4b74-9d22-57adfcb12cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.159997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43328cd-700b-4b74-9d22-57adfcb12cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.160753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43328cd-700b-4b74-9d22-57adfcb12cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:38.162336 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43328cd-700b-4b74-9d22-57adfcb12cfc admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.171107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-618c860b-af74-465e-a49a-941047582cae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cbe3002e3da4cbea85c7a0c02f9f440', 'request_id': 'req-618c860b-af74-465e-a49a-941047582cae'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.173521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 949/1875] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:38.174692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:38.177755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43328cd-700b-4b74-9d22-57adfcb12cfc admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:38.179071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-618c860b-af74-465e-a49a-941047582cae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cbe3002e3da4cbea85c7a0c02f9f440', 'request_id': 'req-618c860b-af74-465e-a49a-941047582cae'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.186866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-618c860b-af74-465e-a49a-941047582cae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cbe3002e3da4cbea85c7a0c02f9f440', 'request_id': 'req-618c860b-af74-465e-a49a-941047582cae'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.189433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43328cd-700b-4b74-9d22-57adfcb12cfc admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:38.198490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-618c860b-af74-465e-a49a-941047582cae admin admin] The token cache is being invalidate because project 7cbe3002e3da4cbea85c7a0c02f9f440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:38.198875 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-618c860b-af74-465e-a49a-941047582cae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.199529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-618c860b-af74-465e-a49a-941047582cae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.201142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 927/1876] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:38 2022] DELETE /identity/v3/projects/7cbe3002e3da4cbea85c7a0c02f9f440 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:38.204881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c860b-af74-465e-a49a-941047582cae admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:38.206223 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-618c860b-af74-465e-a49a-941047582cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:38.255711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb4b42-600b-45d0-9fbd-df4595727784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:38.256796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb4b42-600b-45d0-9fbd-df4595727784 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.257093 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb4b42-600b-45d0-9fbd-df4595727784 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.257378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb4b42-600b-45d0-9fbd-df4595727784 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.258129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb4b42-600b-45d0-9fbd-df4595727784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:38.259900 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb4b42-600b-45d0-9fbd-df4595727784 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.270836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 950/1877] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:38.274850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb4b42-600b-45d0-9fbd-df4595727784 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:38.276773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-02bb4b42-600b-45d0-9fbd-df4595727784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:38.278182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c860b-af74-465e-a49a-941047582cae admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:38.284258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:38.285184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.285464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.285816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] PATH_INFO: `/v3/users/32597b3dc97e46259fa34ce6eefebe0a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.287100 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32597b3dc97e46259fa34ce6eefebe0a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:38.289357 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.294335 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb4b42-600b-45d0-9fbd-df4595727784 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:38.301217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68717607-77b8-415d-819e-e93e388a88db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:38.302242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68717607-77b8-415d-819e-e93e388a88db admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.302536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68717607-77b8-415d-819e-e93e388a88db admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.303000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68717607-77b8-415d-819e-e93e388a88db admin admin] PATH_INFO: `/v3/projects/7eb4a3a2f95046d3be0b8f4abe69f9b8/users/cbc551ed018b402e83b460933698c593/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.320540 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68717607-77b8-415d-819e-e93e388a88db admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eb4a3a2f95046d3be0b8f4abe69f9b8, user_id=cbc551ed018b402e83b460933698c593, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:38.323340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68717607-77b8-415d-819e-e93e388a88db admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.347889 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32597b3dc97e46259fa34ce6eefebe0a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.355883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 951/1878] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:38 2022] PUT /identity/v3/projects/7eb4a3a2f95046d3be0b8f4abe69f9b8/users/cbc551ed018b402e83b460933698c593/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:38.357417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3416ea-f18e-4ad0-bfb2-abf174c9ddef None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.357775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3416ea-f18e-4ad0-bfb2-abf174c9ddef None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.358560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3416ea-f18e-4ad0-bfb2-abf174c9ddef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.362769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32597b3dc97e46259fa34ce6eefebe0a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.380150 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32597b3dc97e46259fa34ce6eefebe0a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.390459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32597b3dc97e46259fa34ce6eefebe0a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.391340 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32597b3dc97e46259fa34ce6eefebe0a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.400986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 928/1879] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:38 2022] DELETE /identity/v3/users/32597b3dc97e46259fa34ce6eefebe0a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:38.409976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-9f3416ea-f18e-4ad0-bfb2-abf174c9ddef None None] MFA Rules not processed for user `cbc551ed018b402e83b460933698c593`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:38.411193 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9f3416ea-f18e-4ad0-bfb2-abf174c9ddef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:38.437206 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:38.438881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:38.485868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:38.495294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 952/1880] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:38.499985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3416ea-f18e-4ad0-bfb2-abf174c9ddef None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:38.502028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9f3416ea-f18e-4ad0-bfb2-abf174c9ddef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:38.513452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184a01b-d031-4196-9f0b-7200a4648589 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:38.519295 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c68705-f16a-426f-8e6f-a29e3f3b39ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:38.520495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3416ea-f18e-4ad0-bfb2-abf174c9ddef None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:38.520699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c68705-f16a-426f-8e6f-a29e3f3b39ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.521053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c68705-f16a-426f-8e6f-a29e3f3b39ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.521332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c68705-f16a-426f-8e6f-a29e3f3b39ab admin admin] PATH_INFO: `/v3/users/9e8677b9ea3a438eba67406eda45f878` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.522610 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c68705-f16a-426f-8e6f-a29e3f3b39ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e8677b9ea3a438eba67406eda45f878)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:38.524589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c68705-f16a-426f-8e6f-a29e3f3b39ab admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.526195 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a64ac7-3122-46ce-8c11-07ab918c8b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:38.526906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a64ac7-3122-46ce-8c11-07ab918c8b45 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.527197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a64ac7-3122-46ce-8c11-07ab918c8b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.527531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a64ac7-3122-46ce-8c11-07ab918c8b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.528239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a64ac7-3122-46ce-8c11-07ab918c8b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:38.528362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:38.530409 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a64ac7-3122-46ce-8c11-07ab918c8b45 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.568786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 953/1881] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:38 2022] POST /identity/v3/projects => generated 399 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:38.573395 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a64ac7-3122-46ce-8c11-07ab918c8b45 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:38.589540 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-00c68705-f16a-426f-8e6f-a29e3f3b39ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e8677b9ea3a438eba67406eda45f878'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.598446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-00c68705-f16a-426f-8e6f-a29e3f3b39ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e8677b9ea3a438eba67406eda45f878'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.615944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-00c68705-f16a-426f-8e6f-a29e3f3b39ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e8677b9ea3a438eba67406eda45f878'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.624163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-00c68705-f16a-426f-8e6f-a29e3f3b39ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e8677b9ea3a438eba67406eda45f878'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.624919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-00c68705-f16a-426f-8e6f-a29e3f3b39ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e8677b9ea3a438eba67406eda45f878'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:38.632902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 929/1882] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:38 2022] DELETE /identity/v3/users/9e8677b9ea3a438eba67406eda45f878 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:38.638462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce1658-4de5-40b7-add1-cf7d2045f21b None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.640096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce1658-4de5-40b7-add1-cf7d2045f21b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.640515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce1658-4de5-40b7-add1-cf7d2045f21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.656384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a64ac7-3122-46ce-8c11-07ab918c8b45 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:38.662292 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-38ce1658-4de5-40b7-add1-cf7d2045f21b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:38.663609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-38ce1658-4de5-40b7-add1-cf7d2045f21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:38.768243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6af93c-19c3-4edf-ba78-3d8e605f959e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:38.769262 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6af93c-19c3-4edf-ba78-3d8e605f959e admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.769692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6af93c-19c3-4edf-ba78-3d8e605f959e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.770208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6af93c-19c3-4edf-ba78-3d8e605f959e admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.771150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6af93c-19c3-4edf-ba78-3d8e605f959e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:38.774280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6af93c-19c3-4edf-ba78-3d8e605f959e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.778143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 930/1883] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:38.790460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce1658-4de5-40b7-add1-cf7d2045f21b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:38.791902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-38ce1658-4de5-40b7-add1-cf7d2045f21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:38.813214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 954/1884] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:38 2022] POST /identity/v3/users => generated 306 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:38.817653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6af93c-19c3-4edf-ba78-3d8e605f959e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:38.820002 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8a6af93c-19c3-4edf-ba78-3d8e605f959e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:38.840998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6af93c-19c3-4edf-ba78-3d8e605f959e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:38.849462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73025251-2d3b-4a6f-b941-0a24f8344b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:38.851384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73025251-2d3b-4a6f-b941-0a24f8344b24 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.852215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73025251-2d3b-4a6f-b941-0a24f8344b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.853241 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73025251-2d3b-4a6f-b941-0a24f8344b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.854216 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73025251-2d3b-4a6f-b941-0a24f8344b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:38.859158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73025251-2d3b-4a6f-b941-0a24f8344b24 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.879029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 955/1885] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:38.881814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73025251-2d3b-4a6f-b941-0a24f8344b24 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:38.907583 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce1658-4de5-40b7-add1-cf7d2045f21b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:38.909362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73025251-2d3b-4a6f-b941-0a24f8344b24 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:38.924832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27839bd-a806-4ee6-804b-fc27b25424fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:38.925510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27839bd-a806-4ee6-804b-fc27b25424fd service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.925851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27839bd-a806-4ee6-804b-fc27b25424fd service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.926140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27839bd-a806-4ee6-804b-fc27b25424fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.942800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c4aa0-befb-4d49-9e3d-95897bbdee73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:38.943882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27839bd-a806-4ee6-804b-fc27b25424fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:38.944628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c4aa0-befb-4d49-9e3d-95897bbdee73 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:38.944923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c4aa0-befb-4d49-9e3d-95897bbdee73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:38.945207 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c4aa0-befb-4d49-9e3d-95897bbdee73 admin admin] PATH_INFO: `/v3/projects/6a4b3b6dd4834965adf25e3a7a4f0076/users/ded1cd942879421f9d9c93bbf8f7d151/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:38.945589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27839bd-a806-4ee6-804b-fc27b25424fd service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.958470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 931/1886] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:46:38.970870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c4aa0-befb-4d49-9e3d-95897bbdee73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4b3b6dd4834965adf25e3a7a4f0076, user_id=ded1cd942879421f9d9c93bbf8f7d151, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:38.974547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c4aa0-befb-4d49-9e3d-95897bbdee73 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:38.995757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 956/1887] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:38 2022] PUT /identity/v3/projects/6a4b3b6dd4834965adf25e3a7a4f0076/users/ded1cd942879421f9d9c93bbf8f7d151/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:38.997793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27839bd-a806-4ee6-804b-fc27b25424fd service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:39.019164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27839bd-a806-4ee6-804b-fc27b25424fd service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:39.063647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:39.121935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b0189-689c-4dd0-8bbc-861d3717d85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:39.122856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b0189-689c-4dd0-8bbc-861d3717d85c admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:39.123621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b0189-689c-4dd0-8bbc-861d3717d85c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:39.123765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b0189-689c-4dd0-8bbc-861d3717d85c admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:39.124758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b0189-689c-4dd0-8bbc-861d3717d85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:39.140862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b0189-689c-4dd0-8bbc-861d3717d85c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:39.169186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 932/1888] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:39.174202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c4aa0-befb-4d49-9e3d-95897bbdee73 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:39.186514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c4aa0-befb-4d49-9e3d-95897bbdee73 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:39.191801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d9a35-d78a-48e8-bec8-1d74060ce8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:39.192746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d9a35-d78a-48e8-bec8-1d74060ce8e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:39.193038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d9a35-d78a-48e8-bec8-1d74060ce8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:39.193627 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d9a35-d78a-48e8-bec8-1d74060ce8e0 admin admin] PATH_INFO: `/v3/projects/6a4b3b6dd4834965adf25e3a7a4f0076/users/ded1cd942879421f9d9c93bbf8f7d151/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:39.199203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d9a35-d78a-48e8-bec8-1d74060ce8e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4b3b6dd4834965adf25e3a7a4f0076, user_id=ded1cd942879421f9d9c93bbf8f7d151, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:39.204603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d9a35-d78a-48e8-bec8-1d74060ce8e0 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:39.227862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 957/1889] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:39 2022] PUT /identity/v3/projects/6a4b3b6dd4834965adf25e3a7a4f0076/users/ded1cd942879421f9d9c93bbf8f7d151/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:39.230901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a1662-0934-4ffb-bb2b-f31054bbee3c None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:39.231276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a1662-0934-4ffb-bb2b-f31054bbee3c None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:39.232342 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a1662-0934-4ffb-bb2b-f31054bbee3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:39.273720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-393a1662-0934-4ffb-bb2b-f31054bbee3c None None] MFA Rules not processed for user `ded1cd942879421f9d9c93bbf8f7d151`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:39.275542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-393a1662-0934-4ffb-bb2b-f31054bbee3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:39.298867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d9a35-d78a-48e8-bec8-1d74060ce8e0 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:39.313486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-bb7d9a35-d78a-48e8-bec8-1d74060ce8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:39.365976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 933/1890] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:39 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:39.369386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:39.377673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a1662-0934-4ffb-bb2b-f31054bbee3c None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:39.379375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-393a1662-0934-4ffb-bb2b-f31054bbee3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:39.427851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d9a35-d78a-48e8-bec8-1d74060ce8e0 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:39.434061 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:39.435071 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:39.435490 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:39.435824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] PATH_INFO: `/v3/projects/8996257b8c53407491875eb792ae3b19` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:39.438809 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8996257b8c53407491875eb792ae3b19)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:39.440874 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:39.477021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a1662-0934-4ffb-bb2b-f31054bbee3c None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:39.482275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be68693-6e29-478e-9027-ead2dd112ca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:39.483680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be68693-6e29-478e-9027-ead2dd112ca9 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:39.484071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be68693-6e29-478e-9027-ead2dd112ca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:39.494626 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be68693-6e29-478e-9027-ead2dd112ca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:39.508233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be68693-6e29-478e-9027-ead2dd112ca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:39.512696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be68693-6e29-478e-9027-ead2dd112ca9 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:39.522088 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8996257b8c53407491875eb792ae3b19', 'request_id': 'req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:39.526703 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 934/1891] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:39 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:46:39.531537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8996257b8c53407491875eb792ae3b19', 'request_id': 'req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:39.539976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8996257b8c53407491875eb792ae3b19', 'request_id': 'req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:39.557122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] The token cache is being invalidate because project 8996257b8c53407491875eb792ae3b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:39.558422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:39.564966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:39.570336 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 958/1892] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:39 2022] DELETE /identity/v3/projects/8996257b8c53407491875eb792ae3b19 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:39.576992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220978b6-d7ff-46cc-bfe4-d97917378b3f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:39.577633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220978b6-d7ff-46cc-bfe4-d97917378b3f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:39.578964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220978b6-d7ff-46cc-bfe4-d97917378b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:39.617579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-220978b6-d7ff-46cc-bfe4-d97917378b3f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:39.624927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-220978b6-d7ff-46cc-bfe4-d97917378b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:39.634427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:39.636387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:39.695945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:39.710167 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177c598-a783-4ea4-a4c0-ecb2a5b2ec39 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:39.715604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca1d15-0cef-43e8-8d40-8c515607f01e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:39.716270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca1d15-0cef-43e8-8d40-8c515607f01e service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:39.716557 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca1d15-0cef-43e8-8d40-8c515607f01e service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:39.716952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca1d15-0cef-43e8-8d40-8c515607f01e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:39.720615 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 935/1893] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:39.727151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64c379-ac24-4d90-a478-de75fe63f6bb None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:39.727687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64c379-ac24-4d90-a478-de75fe63f6bb None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:39.728780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c8ca1d15-0cef-43e8-8d40-8c515607f01e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:39.729233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64c379-ac24-4d90-a478-de75fe63f6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:39.772624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-cc64c379-ac24-4d90-a478-de75fe63f6bb None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:39.773847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-cc64c379-ac24-4d90-a478-de75fe63f6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:39.790795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 936/1894] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:39.792439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64c379-ac24-4d90-a478-de75fe63f6bb None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:39.828818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64c379-ac24-4d90-a478-de75fe63f6bb None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:39.837271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4115ebb-f495-452a-8e89-915d2d3bbad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:39.837962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4115ebb-f495-452a-8e89-915d2d3bbad8 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:39.838236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4115ebb-f495-452a-8e89-915d2d3bbad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:39.838514 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4115ebb-f495-452a-8e89-915d2d3bbad8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:39.839251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4115ebb-f495-452a-8e89-915d2d3bbad8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:39.839556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4115ebb-f495-452a-8e89-915d2d3bbad8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:39.841196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4115ebb-f495-452a-8e89-915d2d3bbad8 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:39.850613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca1d15-0cef-43e8-8d40-8c515607f01e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:39.853095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca1d15-0cef-43e8-8d40-8c515607f01e service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:39.855982 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 937/1895] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:46:39.861965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4115ebb-f495-452a-8e89-915d2d3bbad8 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:39.868457 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 959/1896] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:39.883566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4115ebb-f495-452a-8e89-915d2d3bbad8 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:39.898869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e30cf3-f25b-4820-bbdf-47e64f3d0b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:39.900195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e30cf3-f25b-4820-bbdf-47e64f3d0b88 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:39.900817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e30cf3-f25b-4820-bbdf-47e64f3d0b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:39.901284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e30cf3-f25b-4820-bbdf-47e64f3d0b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:39.902200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e30cf3-f25b-4820-bbdf-47e64f3d0b88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:39.905420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e30cf3-f25b-4820-bbdf-47e64f3d0b88 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:39.962042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 938/1897] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:39 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:39.965911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca1d15-0cef-43e8-8d40-8c515607f01e service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:39.981469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca1d15-0cef-43e8-8d40-8c515607f01e service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:39.991102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:40.101556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866ec17-6c44-443a-850b-d790a9865db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:40.102547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866ec17-6c44-443a-850b-d790a9865db2 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:40.102960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866ec17-6c44-443a-850b-d790a9865db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:40.103382 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866ec17-6c44-443a-850b-d790a9865db2 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:40.104266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b866ec17-6c44-443a-850b-d790a9865db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:40.107210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b866ec17-6c44-443a-850b-d790a9865db2 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:40.150074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 960/1898] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:39 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:40.150755 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e30cf3-f25b-4820-bbdf-47e64f3d0b88 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:40.166134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e30cf3-f25b-4820-bbdf-47e64f3d0b88 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:40.173168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca5b52-f411-4745-9ee3-3cbfd5ff7951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:40.174208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca5b52-f411-4745-9ee3-3cbfd5ff7951 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:40.174736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca5b52-f411-4745-9ee3-3cbfd5ff7951 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:40.175222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca5b52-f411-4745-9ee3-3cbfd5ff7951 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:40.176166 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca5b52-f411-4745-9ee3-3cbfd5ff7951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:40.178040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca5b52-f411-4745-9ee3-3cbfd5ff7951 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:40.191190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 939/1899] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:40.196026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866ec17-6c44-443a-850b-d790a9865db2 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:40.216758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866ec17-6c44-443a-850b-d790a9865db2 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:40.228871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e013c5-f5c8-4c25-bfea-5090c2861cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:40.231806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e013c5-f5c8-4c25-bfea-5090c2861cfa admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:40.232260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e013c5-f5c8-4c25-bfea-5090c2861cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:40.232440 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e013c5-f5c8-4c25-bfea-5090c2861cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:40.233482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e013c5-f5c8-4c25-bfea-5090c2861cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:40.235292 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e013c5-f5c8-4c25-bfea-5090c2861cfa admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:40.254403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 961/1900] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:40.256094 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca5b52-f411-4745-9ee3-3cbfd5ff7951 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:40.271134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca5b52-f411-4745-9ee3-3cbfd5ff7951 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:40.297405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9152f-4d37-4c5a-a2b9-5e020189d4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:40.298541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9152f-4d37-4c5a-a2b9-5e020189d4bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:40.298925 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9152f-4d37-4c5a-a2b9-5e020189d4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:40.299302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9152f-4d37-4c5a-a2b9-5e020189d4bb admin admin] PATH_INFO: `/v3/projects/782e253a9b8742ad828728b133da064a/users/755cf6d4b0f443f6b4b70b5c6843499f/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:40.325885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9152f-4d37-4c5a-a2b9-5e020189d4bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=782e253a9b8742ad828728b133da064a, user_id=755cf6d4b0f443f6b4b70b5c6843499f, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:40.329732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9152f-4d37-4c5a-a2b9-5e020189d4bb admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:40.355567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 940/1901] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:40 2022] PUT /identity/v3/projects/782e253a9b8742ad828728b133da064a/users/755cf6d4b0f443f6b4b70b5c6843499f/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:40.368316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e6edc-6bfe-4534-84fc-25c08179f79d None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:40.368670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e6edc-6bfe-4534-84fc-25c08179f79d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:40.368925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e6edc-6bfe-4534-84fc-25c08179f79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:40.402182 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-533e6edc-6bfe-4534-84fc-25c08179f79d None None] MFA Rules not processed for user `755cf6d4b0f443f6b4b70b5c6843499f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:40.403411 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-533e6edc-6bfe-4534-84fc-25c08179f79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:40.477551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 962/1902] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:40.484847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9152f-4d37-4c5a-a2b9-5e020189d4bb admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:40.523422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e6edc-6bfe-4534-84fc-25c08179f79d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:40.530252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-533e6edc-6bfe-4534-84fc-25c08179f79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:40.586600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9152f-4d37-4c5a-a2b9-5e020189d4bb admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:40.592832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71add7-6204-4026-a568-902444718604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:40.597106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71add7-6204-4026-a568-902444718604 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:40.597875 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71add7-6204-4026-a568-902444718604 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:40.598176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71add7-6204-4026-a568-902444718604 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:40.601293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71add7-6204-4026-a568-902444718604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:40.603241 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71add7-6204-4026-a568-902444718604 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:40.638664 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e6edc-6bfe-4534-84fc-25c08179f79d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:40.646169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 941/1903] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:40 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:40.647637 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71add7-6204-4026-a568-902444718604 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:40.647978 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:40.649138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:40.649774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:40.650259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] PATH_INFO: `/v3/projects/908a6c6784b14cffb4498e48f0c393be` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:40.652084 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=908a6c6784b14cffb4498e48f0c393be)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:40.655297 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:40.681884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:40.730795 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '908a6c6784b14cffb4498e48f0c393be', 'request_id': 'req-33fccce4-80e1-415a-a8c4-34bb48dafcd4'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:40.741479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '908a6c6784b14cffb4498e48f0c393be', 'request_id': 'req-33fccce4-80e1-415a-a8c4-34bb48dafcd4'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:40.749944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '908a6c6784b14cffb4498e48f0c393be', 'request_id': 'req-33fccce4-80e1-415a-a8c4-34bb48dafcd4'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:40.751473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:40.755726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71add7-6204-4026-a568-902444718604 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:40.765906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] The token cache is being invalidate because project 908a6c6784b14cffb4498e48f0c393be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:40.766205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:40.766975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:40.771892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 963/1904] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:40 2022] DELETE /identity/v3/projects/908a6c6784b14cffb4498e48f0c393be => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:40.774185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:40.775758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:40.835367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e55657-6aa1-4d03-a08e-ee288e18be94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:40.836059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e55657-6aa1-4d03-a08e-ee288e18be94 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:40.836346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e55657-6aa1-4d03-a08e-ee288e18be94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:40.836662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e55657-6aa1-4d03-a08e-ee288e18be94 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:40.837377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e55657-6aa1-4d03-a08e-ee288e18be94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:40.839327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e55657-6aa1-4d03-a08e-ee288e18be94 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:40.862306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fccce4-80e1-415a-a8c4-34bb48dafcd4 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:40.866396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 942/1905] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:40 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:40.867440 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:40.868334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:40.868642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:40.868839 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e55657-6aa1-4d03-a08e-ee288e18be94 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:40.868937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] PATH_INFO: `/v3/users/d5c31ef3f425476087f5b6e7181e0352` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:40.870139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c31ef3f425476087f5b6e7181e0352)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:40.870603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b4e55657-6aa1-4d03-a08e-ee288e18be94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:40.872069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:40.884153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e55657-6aa1-4d03-a08e-ee288e18be94 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:40.890083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed671c-14b6-4968-809a-9a418168b8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:40.890692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed671c-14b6-4968-809a-9a418168b8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:40.890979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed671c-14b6-4968-809a-9a418168b8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:40.891258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed671c-14b6-4968-809a-9a418168b8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:40.892005 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed671c-14b6-4968-809a-9a418168b8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:40.893823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed671c-14b6-4968-809a-9a418168b8a3 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:40.906106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 943/1906] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:40.910262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed671c-14b6-4968-809a-9a418168b8a3 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:40.919931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed671c-14b6-4968-809a-9a418168b8a3 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:40.924817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3397-9977-42cf-80ec-7d883a634c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:40.925701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3397-9977-42cf-80ec-7d883a634c3a admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:40.925988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3397-9977-42cf-80ec-7d883a634c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:40.926466 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3397-9977-42cf-80ec-7d883a634c3a admin admin] PATH_INFO: `/v3/projects/ae852b6fafeb47219038302950a98c4f/users/6179a20ada84486eb307e1362529c981/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:40.927754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c31ef3f425476087f5b6e7181e0352'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:40.928419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c31ef3f425476087f5b6e7181e0352'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:40.935503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c31ef3f425476087f5b6e7181e0352'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:40.942270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a3397-9977-42cf-80ec-7d883a634c3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae852b6fafeb47219038302950a98c4f, user_id=6179a20ada84486eb307e1362529c981, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:40.947653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a3397-9977-42cf-80ec-7d883a634c3a admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:40.948467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c31ef3f425476087f5b6e7181e0352'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:40.956450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c31ef3f425476087f5b6e7181e0352'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:40.962722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 964/1907] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:40 2022] DELETE /identity/v3/users/d5c31ef3f425476087f5b6e7181e0352 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:40.963540 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 944/1908] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:40 2022] PUT /identity/v3/projects/ae852b6fafeb47219038302950a98c4f/users/6179a20ada84486eb307e1362529c981/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:40.967966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:40.969480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:41.004639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:41.058464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798e169-0bd2-4cbc-9ed9-4c6a369ec919 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:41.065574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea6ede-704b-425a-999e-4f4d8af9f89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:41.067408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea6ede-704b-425a-999e-4f4d8af9f89a admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.067673 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea6ede-704b-425a-999e-4f4d8af9f89a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.067951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea6ede-704b-425a-999e-4f4d8af9f89a admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.068956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea6ede-704b-425a-999e-4f4d8af9f89a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:41.070570 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea6ede-704b-425a-999e-4f4d8af9f89a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:41.083925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 965/1909] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:41.088522 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3397-9977-42cf-80ec-7d883a634c3a admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:41.171806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3397-9977-42cf-80ec-7d883a634c3a admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:41.178040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47266600-f061-4f00-af18-e5d4385ed64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:41.178948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47266600-f061-4f00-af18-e5d4385ed64a admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.179281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47266600-f061-4f00-af18-e5d4385ed64a admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.179560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47266600-f061-4f00-af18-e5d4385ed64a admin admin] PATH_INFO: `/v3/projects/ae852b6fafeb47219038302950a98c4f/users/6179a20ada84486eb307e1362529c981/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.182372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47266600-f061-4f00-af18-e5d4385ed64a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae852b6fafeb47219038302950a98c4f, user_id=6179a20ada84486eb307e1362529c981, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:41.185560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47266600-f061-4f00-af18-e5d4385ed64a admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:41.200637 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 945/1910] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:41 2022] PUT /identity/v3/projects/ae852b6fafeb47219038302950a98c4f/users/6179a20ada84486eb307e1362529c981/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:41.210882 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0c63f-1cc1-4033-b9bd-9f83ea386184 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.211181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0c63f-1cc1-4033-b9bd-9f83ea386184 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.212247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0c63f-1cc1-4033-b9bd-9f83ea386184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.251884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7eb0c63f-1cc1-4033-b9bd-9f83ea386184 None None] MFA Rules not processed for user `6179a20ada84486eb307e1362529c981`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:41.253639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7eb0c63f-1cc1-4033-b9bd-9f83ea386184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:41.269503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47266600-f061-4f00-af18-e5d4385ed64a admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:41.271223 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-47266600-f061-4f00-af18-e5d4385ed64a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:41.351935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 966/1911] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:41.358631 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:41.370863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0c63f-1cc1-4033-b9bd-9f83ea386184 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:41.371894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47266600-f061-4f00-af18-e5d4385ed64a admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:41.372518 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7eb0c63f-1cc1-4033-b9bd-9f83ea386184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:41.378836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:41.379898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.380232 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.380662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e admin admin] PATH_INFO: `/v3/projects/89beadda94d145338daf12626737c568` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.382055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=89beadda94d145338daf12626737c568)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:41.384651 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:41.444391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89beadda94d145338daf12626737c568', 'request_id': 'req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:41.445846 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0c63f-1cc1-4033-b9bd-9f83ea386184 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:41.456266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89beadda94d145338daf12626737c568', 'request_id': 'req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:41.463027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89beadda94d145338daf12626737c568', 'request_id': 'req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:41.477143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e admin admin] The token cache is being invalidate because project 89beadda94d145338daf12626737c568 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:41.477641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:41.478608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-5d83ffd6-5b1d-49eb-b931-eb9005eabd8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:41.481294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 946/1912] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:41 2022] DELETE /identity/v3/projects/89beadda94d145338daf12626737c568 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:41.486535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cb3e7-4e87-4545-92b3-ddb4c14cf390 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.487400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cb3e7-4e87-4545-92b3-ddb4c14cf390 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.488042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cb3e7-4e87-4545-92b3-ddb4c14cf390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.511109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-957cb3e7-4e87-4545-92b3-ddb4c14cf390 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:41.512335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-957cb3e7-4e87-4545-92b3-ddb4c14cf390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:41.520255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b364a81-3b13-4caa-bf3e-78dfc0c7ecb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:41.520957 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b364a81-3b13-4caa-bf3e-78dfc0c7ecb5 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.521241 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b364a81-3b13-4caa-bf3e-78dfc0c7ecb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.521521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b364a81-3b13-4caa-bf3e-78dfc0c7ecb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.523326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1b364a81-3b13-4caa-bf3e-78dfc0c7ecb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:41.544182 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:41.580367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 947/1913] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:41.589219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3c84d-9dcc-4566-883c-0debb517332a None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.589510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3c84d-9dcc-4566-883c-0debb517332a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.589786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3c84d-9dcc-4566-883c-0debb517332a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.597860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b364a81-3b13-4caa-bf3e-78dfc0c7ecb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:41.599311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b364a81-3b13-4caa-bf3e-78dfc0c7ecb5 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:41.608975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 967/1914] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:46:41.609187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-e9e3c84d-9dcc-4566-883c-0debb517332a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:41.610320 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e9e3c84d-9dcc-4566-883c-0debb517332a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:41.618389 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 948/1915] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:41.622488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b364a81-3b13-4caa-bf3e-78dfc0c7ecb5 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:41.632857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b364a81-3b13-4caa-bf3e-78dfc0c7ecb5 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:41.638629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327958e-3788-4d69-8f2e-60434a37a71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:41.639937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327958e-3788-4d69-8f2e-60434a37a71a admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.640313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327958e-3788-4d69-8f2e-60434a37a71a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.640611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327958e-3788-4d69-8f2e-60434a37a71a admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.641334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327958e-3788-4d69-8f2e-60434a37a71a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:41.641658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327958e-3788-4d69-8f2e-60434a37a71a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:41.643443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327958e-3788-4d69-8f2e-60434a37a71a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:41.657450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 968/1916] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:46:41.661629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3c84d-9dcc-4566-883c-0debb517332a None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:41.672053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3c84d-9dcc-4566-883c-0debb517332a None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:41.676705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aebbbc-c3e0-4636-b30e-5290e7b8f0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:41.677358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aebbbc-c3e0-4636-b30e-5290e7b8f0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.677640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aebbbc-c3e0-4636-b30e-5290e7b8f0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.677919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aebbbc-c3e0-4636-b30e-5290e7b8f0a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.678662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aebbbc-c3e0-4636-b30e-5290e7b8f0a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:41.681148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aebbbc-c3e0-4636-b30e-5290e7b8f0a2 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:41.705856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327958e-3788-4d69-8f2e-60434a37a71a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:41.708064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4327958e-3788-4d69-8f2e-60434a37a71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:41.716809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 949/1917] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:41 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:41.719003 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aebbbc-c3e0-4636-b30e-5290e7b8f0a2 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:41.721104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327958e-3788-4d69-8f2e-60434a37a71a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:41.777087 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365dd4a-d9ce-4091-a27a-91c0d86e76fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:41.777788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365dd4a-d9ce-4091-a27a-91c0d86e76fb service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.778079 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365dd4a-d9ce-4091-a27a-91c0d86e76fb service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.778461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365dd4a-d9ce-4091-a27a-91c0d86e76fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.780434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a365dd4a-d9ce-4091-a27a-91c0d86e76fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:41.781068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:41.786668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aebbbc-c3e0-4636-b30e-5290e7b8f0a2 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:41.793288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb91e1f-a1ee-44c7-8410-cb520f6b396b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:41.794105 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb91e1f-a1ee-44c7-8410-cb520f6b396b admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.794895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb91e1f-a1ee-44c7-8410-cb520f6b396b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.795826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb91e1f-a1ee-44c7-8410-cb520f6b396b admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.796547 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb91e1f-a1ee-44c7-8410-cb520f6b396b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:41.799039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb91e1f-a1ee-44c7-8410-cb520f6b396b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:41.824606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 950/1918] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:41 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:41.827980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb91e1f-a1ee-44c7-8410-cb520f6b396b admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:41.839458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb91e1f-a1ee-44c7-8410-cb520f6b396b admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:41.845162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33f562-2d2e-4307-93eb-d450a187ed81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:41.845842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33f562-2d2e-4307-93eb-d450a187ed81 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.846130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33f562-2d2e-4307-93eb-d450a187ed81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.846412 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33f562-2d2e-4307-93eb-d450a187ed81 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.847153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33f562-2d2e-4307-93eb-d450a187ed81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:41.848911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33f562-2d2e-4307-93eb-d450a187ed81 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:41.855190 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365dd4a-d9ce-4091-a27a-91c0d86e76fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:41.856743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365dd4a-d9ce-4091-a27a-91c0d86e76fb service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:41.863679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 951/1919] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:41.864545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33f562-2d2e-4307-93eb-d450a187ed81 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:41.867551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 969/1920] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:46:41.874918 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33f562-2d2e-4307-93eb-d450a187ed81 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:41.879855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf685c4a-184d-4ee8-b389-b602f4a826f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:41.880520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf685c4a-184d-4ee8-b389-b602f4a826f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.880853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf685c4a-184d-4ee8-b389-b602f4a826f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.881136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf685c4a-184d-4ee8-b389-b602f4a826f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.881901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf685c4a-184d-4ee8-b389-b602f4a826f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:41.883485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf685c4a-184d-4ee8-b389-b602f4a826f1 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:41.896650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 952/1921] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:41.898774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365dd4a-d9ce-4091-a27a-91c0d86e76fb service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:41.910805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365dd4a-d9ce-4091-a27a-91c0d86e76fb service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:41.915519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435aa5e-6148-4b68-abb0-f0268eb6181e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:41.916429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435aa5e-6148-4b68-abb0-f0268eb6181e admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.916783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435aa5e-6148-4b68-abb0-f0268eb6181e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.917060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435aa5e-6148-4b68-abb0-f0268eb6181e admin admin] PATH_INFO: `/v3/projects/32f9b858351443e7ac78481cb57ce46f/users/52a0337765aa4ea1a2228595e56c9142/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.932634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435aa5e-6148-4b68-abb0-f0268eb6181e admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f9b858351443e7ac78481cb57ce46f, user_id=52a0337765aa4ea1a2228595e56c9142, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:41.935365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435aa5e-6148-4b68-abb0-f0268eb6181e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:41.949620 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 970/1922] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:41 2022] PUT /identity/v3/projects/32f9b858351443e7ac78481cb57ce46f/users/52a0337765aa4ea1a2228595e56c9142/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:41.959898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda6307-671b-4361-8bbd-ae0d7fdcb6d5 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:41.960293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda6307-671b-4361-8bbd-ae0d7fdcb6d5 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:41.960585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda6307-671b-4361-8bbd-ae0d7fdcb6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:41.991729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-fdda6307-671b-4361-8bbd-ae0d7fdcb6d5 None None] MFA Rules not processed for user `52a0337765aa4ea1a2228595e56c9142`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:41.992998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-fdda6307-671b-4361-8bbd-ae0d7fdcb6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:42.070273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 953/1923] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:42.074735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435aa5e-6148-4b68-abb0-f0268eb6181e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:42.110653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:42.139418 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda6307-671b-4361-8bbd-ae0d7fdcb6d5 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:42.140924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-fdda6307-671b-4361-8bbd-ae0d7fdcb6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:42.144177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435aa5e-6148-4b68-abb0-f0268eb6181e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:42.149089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c453959-38dc-472e-85f0-dd9ba9e607d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:42.149784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c453959-38dc-472e-85f0-dd9ba9e607d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:42.150067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c453959-38dc-472e-85f0-dd9ba9e607d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:42.150341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c453959-38dc-472e-85f0-dd9ba9e607d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:42.151050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c453959-38dc-472e-85f0-dd9ba9e607d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:42.152970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c453959-38dc-472e-85f0-dd9ba9e607d8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:42.154124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda6307-671b-4361-8bbd-ae0d7fdcb6d5 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:42.160186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cce43-a1b7-45a8-9509-ee05b746fc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:42.161042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cce43-a1b7-45a8-9509-ee05b746fc8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:42.161328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cce43-a1b7-45a8-9509-ee05b746fc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:42.161603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cce43-a1b7-45a8-9509-ee05b746fc8c admin admin] PATH_INFO: `/v3/users/755cf6d4b0f443f6b4b70b5c6843499f` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:42.162827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10cce43-a1b7-45a8-9509-ee05b746fc8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=755cf6d4b0f443f6b4b70b5c6843499f)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:42.164859 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10cce43-a1b7-45a8-9509-ee05b746fc8c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:42.181463 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 971/1924] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:42 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:42.185416 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c453959-38dc-472e-85f0-dd9ba9e607d8 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:42.212550 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c10cce43-a1b7-45a8-9509-ee05b746fc8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755cf6d4b0f443f6b4b70b5c6843499f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:42.221678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c10cce43-a1b7-45a8-9509-ee05b746fc8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755cf6d4b0f443f6b4b70b5c6843499f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:42.236364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c10cce43-a1b7-45a8-9509-ee05b746fc8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755cf6d4b0f443f6b4b70b5c6843499f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:42.244853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c10cce43-a1b7-45a8-9509-ee05b746fc8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755cf6d4b0f443f6b4b70b5c6843499f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:42.245715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c10cce43-a1b7-45a8-9509-ee05b746fc8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755cf6d4b0f443f6b4b70b5c6843499f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:42.250247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c453959-38dc-472e-85f0-dd9ba9e607d8 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:42.254226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 954/1925] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:42 2022] DELETE /identity/v3/users/755cf6d4b0f443f6b4b70b5c6843499f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:42.256477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65b226-e2c7-4e52-b183-c1e0230910b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:42.257308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65b226-e2c7-4e52-b183-c1e0230910b6 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:42.257598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65b226-e2c7-4e52-b183-c1e0230910b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:42.257875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65b226-e2c7-4e52-b183-c1e0230910b6 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:42.259719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65b226-e2c7-4e52-b183-c1e0230910b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:42.259847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d50c75-2e55-4220-ae52-3ad2f181008b None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:42.259847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d50c75-2e55-4220-ae52-3ad2f181008b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:42.259847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d50c75-2e55-4220-ae52-3ad2f181008b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:42.260783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65b226-e2c7-4e52-b183-c1e0230910b6 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:42.283810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 972/1926] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:42 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:42.285900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-64d50c75-2e55-4220-ae52-3ad2f181008b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:42.287255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-64d50c75-2e55-4220-ae52-3ad2f181008b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:42.287894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65b226-e2c7-4e52-b183-c1e0230910b6 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:42.302157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3e65b226-e2c7-4e52-b183-c1e0230910b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:42.315143 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65b226-e2c7-4e52-b183-c1e0230910b6 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:42.328981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:42.382063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12409c05-7339-456c-b0cf-c8fa0634c9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:42.382907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12409c05-7339-456c-b0cf-c8fa0634c9aa admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:42.383234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12409c05-7339-456c-b0cf-c8fa0634c9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:42.383541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12409c05-7339-456c-b0cf-c8fa0634c9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:42.384306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12409c05-7339-456c-b0cf-c8fa0634c9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:42.385399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 955/1927] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:42.386218 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12409c05-7339-456c-b0cf-c8fa0634c9aa admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:42.394229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d50c75-2e55-4220-ae52-3ad2f181008b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:42.395748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-64d50c75-2e55-4220-ae52-3ad2f181008b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:42.401078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 973/1928] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:42.402428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12409c05-7339-456c-b0cf-c8fa0634c9aa admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:42.414297 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12409c05-7339-456c-b0cf-c8fa0634c9aa admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:42.419654 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215c82c-ac06-4983-a067-5bcdd6d3e1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:42.421106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215c82c-ac06-4983-a067-5bcdd6d3e1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:42.421400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215c82c-ac06-4983-a067-5bcdd6d3e1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:42.421682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215c82c-ac06-4983-a067-5bcdd6d3e1bc admin admin] PATH_INFO: `/v3/projects/a089850e154e4df79765546bf55a3d52/users/50a95c5f628f4835af28afce075fa97d/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:42.450871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215c82c-ac06-4983-a067-5bcdd6d3e1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a089850e154e4df79765546bf55a3d52, user_id=50a95c5f628f4835af28afce075fa97d, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:42.453972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215c82c-ac06-4983-a067-5bcdd6d3e1bc admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:42.462605 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d50c75-2e55-4220-ae52-3ad2f181008b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:42.468971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 974/1929] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:42 2022] PUT /identity/v3/projects/a089850e154e4df79765546bf55a3d52/users/50a95c5f628f4835af28afce075fa97d/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:42.474517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215c82c-ac06-4983-a067-5bcdd6d3e1bc admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:42.494101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:42.515600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8f376-5660-4afc-8915-d65cb7a38bb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:42.516259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8f376-5660-4afc-8915-d65cb7a38bb9 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:42.516543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8f376-5660-4afc-8915-d65cb7a38bb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:42.516851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8f376-5660-4afc-8915-d65cb7a38bb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:42.540426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215c82c-ac06-4983-a067-5bcdd6d3e1bc admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:42.547435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f8e5d-977a-4f60-a85f-fda447d58529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:42.548117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f8e5d-977a-4f60-a85f-fda447d58529 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:42.548458 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f8e5d-977a-4f60-a85f-fda447d58529 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:42.548769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f8e5d-977a-4f60-a85f-fda447d58529 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:42.549505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f8e5d-977a-4f60-a85f-fda447d58529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:42.551083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f8e5d-977a-4f60-a85f-fda447d58529 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:42.569318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 975/1930] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:42.573970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f8e5d-977a-4f60-a85f-fda447d58529 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:42.584213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f8e5d-977a-4f60-a85f-fda447d58529 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:42.589102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895be95f-ab06-4151-8116-a2848a869893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:42.589998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895be95f-ab06-4151-8116-a2848a869893 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:42.590280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895be95f-ab06-4151-8116-a2848a869893 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:42.590556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895be95f-ab06-4151-8116-a2848a869893 admin admin] PATH_INFO: `/v3/projects/a089850e154e4df79765546bf55a3d52/users/50a95c5f628f4835af28afce075fa97d/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:42.592626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895be95f-ab06-4151-8116-a2848a869893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a089850e154e4df79765546bf55a3d52, user_id=50a95c5f628f4835af28afce075fa97d, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:42.595452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895be95f-ab06-4151-8116-a2848a869893 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:42.598529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8f376-5660-4afc-8915-d65cb7a38bb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:42.600145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8f376-5660-4afc-8915-d65cb7a38bb9 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:42.606184 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 976/1931] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:42 2022] PUT /identity/v3/projects/a089850e154e4df79765546bf55a3d52/users/50a95c5f628f4835af28afce075fa97d/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:42.611162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 956/1932] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:46:42.617333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403a687-e121-42ee-803b-b491b9d15380 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:42.617624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403a687-e121-42ee-803b-b491b9d15380 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:42.617908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403a687-e121-42ee-803b-b491b9d15380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:42.650674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-6403a687-e121-42ee-803b-b491b9d15380 None None] MFA Rules not processed for user `50a95c5f628f4835af28afce075fa97d`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:42.651968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6403a687-e121-42ee-803b-b491b9d15380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:42.736463 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 977/1933] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:42 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:42.754142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8f376-5660-4afc-8915-d65cb7a38bb9 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:42.755637 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-23c8f376-5660-4afc-8915-d65cb7a38bb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:42.811662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:42.837145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8f376-5660-4afc-8915-d65cb7a38bb9 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:42.892559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92523d-a4d0-4e23-afa5-3028db9afb3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:42.893253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92523d-a4d0-4e23-afa5-3028db9afb3f service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:42.893535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92523d-a4d0-4e23-afa5-3028db9afb3f service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:42.893810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92523d-a4d0-4e23-afa5-3028db9afb3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:42.908050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92523d-a4d0-4e23-afa5-3028db9afb3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:42.912785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92523d-a4d0-4e23-afa5-3028db9afb3f service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:42.923969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 957/1934] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:42 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:42.925477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403a687-e121-42ee-803b-b491b9d15380 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:42.926976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6403a687-e121-42ee-803b-b491b9d15380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:43.039164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403a687-e121-42ee-803b-b491b9d15380 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:43.049875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:43.050637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:43.050926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:43.051207 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] PATH_INFO: `/v3/projects/782e253a9b8742ad828728b133da064a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:43.052561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=782e253a9b8742ad828728b133da064a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:43.055588 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:43.110538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:43.134559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '782e253a9b8742ad828728b133da064a', 'request_id': 'req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:43.147961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '782e253a9b8742ad828728b133da064a', 'request_id': 'req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:43.157942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '782e253a9b8742ad828728b133da064a', 'request_id': 'req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:43.173323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] The token cache is being invalidate because project 782e253a9b8742ad828728b133da064a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:43.173323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:43.174170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:43.176044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 978/1935] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:42 2022] DELETE /identity/v3/projects/782e253a9b8742ad828728b133da064a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:43.180021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92523d-a4d0-4e23-afa5-3028db9afb3f service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:43.191501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92523d-a4d0-4e23-afa5-3028db9afb3f service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:43.262286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e6e79-9a98-4e44-8b8d-67edede29e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:43.263477 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e6e79-9a98-4e44-8b8d-67edede29e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:43.263924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e6e79-9a98-4e44-8b8d-67edede29e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:43.264203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e6e79-9a98-4e44-8b8d-67edede29e1b admin admin] PATH_INFO: `/v3/users/6179a20ada84486eb307e1362529c981` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:43.265499 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e6e79-9a98-4e44-8b8d-67edede29e1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6179a20ada84486eb307e1362529c981)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:43.267620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e6e79-9a98-4e44-8b8d-67edede29e1b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:43.332675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-da7e6e79-9a98-4e44-8b8d-67edede29e1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6179a20ada84486eb307e1362529c981'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:43.347869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-da7e6e79-9a98-4e44-8b8d-67edede29e1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6179a20ada84486eb307e1362529c981'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:43.374586 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-da7e6e79-9a98-4e44-8b8d-67edede29e1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6179a20ada84486eb307e1362529c981'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:43.382544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-da7e6e79-9a98-4e44-8b8d-67edede29e1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6179a20ada84486eb307e1362529c981'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:43.383392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-da7e6e79-9a98-4e44-8b8d-67edede29e1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6179a20ada84486eb307e1362529c981'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:43.390578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 958/1936] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:43 2022] DELETE /identity/v3/users/6179a20ada84486eb307e1362529c981 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:43.733592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:43.734507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:43.748902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09f0f6-927a-4314-a7c2-7e23597d3b0f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:43.868819 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9d503-3bc6-49a0-9f4a-894d88f696bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:43.869864 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9d503-3bc6-49a0-9f4a-894d88f696bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:43.870252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9d503-3bc6-49a0-9f4a-894d88f696bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:43.870641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9d503-3bc6-49a0-9f4a-894d88f696bb admin admin] PATH_INFO: `/v3/projects/ae852b6fafeb47219038302950a98c4f` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:43.872175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9d503-3bc6-49a0-9f4a-894d88f696bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae852b6fafeb47219038302950a98c4f)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:43.875083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9d503-3bc6-49a0-9f4a-894d88f696bb admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:43.961642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-35a9d503-3bc6-49a0-9f4a-894d88f696bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae852b6fafeb47219038302950a98c4f', 'request_id': 'req-35a9d503-3bc6-49a0-9f4a-894d88f696bb'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:43.994202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-35a9d503-3bc6-49a0-9f4a-894d88f696bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae852b6fafeb47219038302950a98c4f', 'request_id': 'req-35a9d503-3bc6-49a0-9f4a-894d88f696bb'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:44.005194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-35a9d503-3bc6-49a0-9f4a-894d88f696bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae852b6fafeb47219038302950a98c4f', 'request_id': 'req-35a9d503-3bc6-49a0-9f4a-894d88f696bb'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:44.020018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-35a9d503-3bc6-49a0-9f4a-894d88f696bb admin admin] The token cache is being invalidate because project ae852b6fafeb47219038302950a98c4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:44.020391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-35a9d503-3bc6-49a0-9f4a-894d88f696bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:44.021232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-35a9d503-3bc6-49a0-9f4a-894d88f696bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:44.026632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 979/1937] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:43 2022] DELETE /identity/v3/projects/ae852b6fafeb47219038302950a98c4f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:44.028248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b221b85-ce56-400e-9ba4-ef908c0c157f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:44.028743 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b221b85-ce56-400e-9ba4-ef908c0c157f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:44.028867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b221b85-ce56-400e-9ba4-ef908c0c157f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:44.052304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3b221b85-ce56-400e-9ba4-ef908c0c157f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:44.053902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3b221b85-ce56-400e-9ba4-ef908c0c157f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:44.063649 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:44.204739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 959/1938] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:44.290427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c8a9b-f016-44b1-94fc-b1b729d3df01 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:44.291061 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c8a9b-f016-44b1-94fc-b1b729d3df01 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:44.291475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c8a9b-f016-44b1-94fc-b1b729d3df01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:44.334088 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-808c8a9b-f016-44b1-94fc-b1b729d3df01 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:44.335349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-808c8a9b-f016-44b1-94fc-b1b729d3df01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:44.351692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 980/1939] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:44.352895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b221b85-ce56-400e-9ba4-ef908c0c157f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:44.382565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b221b85-ce56-400e-9ba4-ef908c0c157f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:44.406461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54252766-fc9f-412e-94a0-65845183d9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:44.407319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54252766-fc9f-412e-94a0-65845183d9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:44.410061 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54252766-fc9f-412e-94a0-65845183d9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:44.410794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54252766-fc9f-412e-94a0-65845183d9e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:44.411901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54252766-fc9f-412e-94a0-65845183d9e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:44.412225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54252766-fc9f-412e-94a0-65845183d9e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:44.415048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54252766-fc9f-412e-94a0-65845183d9e7 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:44.432988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 960/1940] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:46:44.437245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c8a9b-f016-44b1-94fc-b1b729d3df01 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:44.455087 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c8a9b-f016-44b1-94fc-b1b729d3df01 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:44.474447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a313322-8a1f-4586-a822-3617a804b174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:44.477792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a313322-8a1f-4586-a822-3617a804b174 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:44.478095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a313322-8a1f-4586-a822-3617a804b174 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:44.478380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a313322-8a1f-4586-a822-3617a804b174 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:44.480081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a313322-8a1f-4586-a822-3617a804b174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:44.483507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a313322-8a1f-4586-a822-3617a804b174 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:44.569405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 981/1941] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:44 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:44.570643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54252766-fc9f-412e-94a0-65845183d9e7 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:44.585045 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54252766-fc9f-412e-94a0-65845183d9e7 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:44.705641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512270a-dbcf-4dd4-85a0-855ce5e6af37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:44.706415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512270a-dbcf-4dd4-85a0-855ce5e6af37 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:44.706778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512270a-dbcf-4dd4-85a0-855ce5e6af37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:44.707283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512270a-dbcf-4dd4-85a0-855ce5e6af37 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:44.709160 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512270a-dbcf-4dd4-85a0-855ce5e6af37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:44.711329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512270a-dbcf-4dd4-85a0-855ce5e6af37 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:44.749815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 961/1942] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:44 2022] POST /identity/v3/users => generated 287 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:44.750965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a313322-8a1f-4586-a822-3617a804b174 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:44.780766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a313322-8a1f-4586-a822-3617a804b174 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:44.793780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9b022-575f-4de9-9c2b-702b55df99f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:44.797810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9b022-575f-4de9-9c2b-702b55df99f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:44.798498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9b022-575f-4de9-9c2b-702b55df99f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:44.799041 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9b022-575f-4de9-9c2b-702b55df99f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:44.800039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9b022-575f-4de9-9c2b-702b55df99f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:44.804962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9b022-575f-4de9-9c2b-702b55df99f9 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:44.846021 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 982/1943] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:44.853786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512270a-dbcf-4dd4-85a0-855ce5e6af37 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:44.869105 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512270a-dbcf-4dd4-85a0-855ce5e6af37 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:44.880798 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610312e-4b39-45ea-85f6-106289ebd4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:44.881885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610312e-4b39-45ea-85f6-106289ebd4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:44.882410 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610312e-4b39-45ea-85f6-106289ebd4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:44.882858 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610312e-4b39-45ea-85f6-106289ebd4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:44.884106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610312e-4b39-45ea-85f6-106289ebd4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:44.889034 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610312e-4b39-45ea-85f6-106289ebd4c1 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:44.925673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 962/1944] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:44.929669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9b022-575f-4de9-9c2b-702b55df99f9 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:44.945150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9b022-575f-4de9-9c2b-702b55df99f9 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:44.960915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b626ef7-38a8-4f08-8223-3eb225aa0dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:44.962277 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b626ef7-38a8-4f08-8223-3eb225aa0dd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:44.962853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b626ef7-38a8-4f08-8223-3eb225aa0dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:44.963000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b626ef7-38a8-4f08-8223-3eb225aa0dd8 admin admin] PATH_INFO: `/v3/projects/d358c71209624c55b7db08cc3435dbcb/users/bf0c0131372345e6b23260b466ec8ff9/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:44.970673 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:45.006290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b626ef7-38a8-4f08-8223-3eb225aa0dd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d358c71209624c55b7db08cc3435dbcb, user_id=bf0c0131372345e6b23260b466ec8ff9, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:45.013544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b626ef7-38a8-4f08-8223-3eb225aa0dd8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:45.038319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 983/1945] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:44 2022] PUT /identity/v3/projects/d358c71209624c55b7db08cc3435dbcb/users/bf0c0131372345e6b23260b466ec8ff9/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:45.057307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573fc56-9fcc-4e08-b2cb-dc5ff069a55f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:45.057307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573fc56-9fcc-4e08-b2cb-dc5ff069a55f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:45.057307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573fc56-9fcc-4e08-b2cb-dc5ff069a55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:45.078205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-7573fc56-9fcc-4e08-b2cb-dc5ff069a55f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:45.079857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7573fc56-9fcc-4e08-b2cb-dc5ff069a55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:45.173416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 963/1946] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:45.186295 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b626ef7-38a8-4f08-8223-3eb225aa0dd8 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:45.188943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9b626ef7-38a8-4f08-8223-3eb225aa0dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:45.311273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b626ef7-38a8-4f08-8223-3eb225aa0dd8 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:45.319531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f37feb-1d64-472b-b61f-ca147ebd692c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:45.321385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f37feb-1d64-472b-b61f-ca147ebd692c service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:45.321845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f37feb-1d64-472b-b61f-ca147ebd692c service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:45.322495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f37feb-1d64-472b-b61f-ca147ebd692c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:45.335439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f37feb-1d64-472b-b61f-ca147ebd692c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:45.337796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f37feb-1d64-472b-b61f-ca147ebd692c service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:45.354438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 984/1947] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:45.509628 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573fc56-9fcc-4e08-b2cb-dc5ff069a55f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:45.511362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7573fc56-9fcc-4e08-b2cb-dc5ff069a55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:45.524878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573fc56-9fcc-4e08-b2cb-dc5ff069a55f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:45.530676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:45.531623 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:45.531909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:45.532280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] PATH_INFO: `/v3/users/52a0337765aa4ea1a2228595e56c9142` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:45.533804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52a0337765aa4ea1a2228595e56c9142)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:45.541634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:45.558714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:45.630993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52a0337765aa4ea1a2228595e56c9142'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:45.641619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52a0337765aa4ea1a2228595e56c9142'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:45.662153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52a0337765aa4ea1a2228595e56c9142'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:45.683404 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52a0337765aa4ea1a2228595e56c9142'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:45.685059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52a0337765aa4ea1a2228595e56c9142'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:45.704453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 964/1948] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:45 2022] DELETE /identity/v3/users/52a0337765aa4ea1a2228595e56c9142 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:45.709100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e19e8-7043-486f-ab7b-0cd7b515a051 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:45.709416 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e19e8-7043-486f-ab7b-0cd7b515a051 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:45.709699 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e19e8-7043-486f-ab7b-0cd7b515a051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:45.731565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-f70e19e8-7043-486f-ab7b-0cd7b515a051 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:45.733145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f70e19e8-7043-486f-ab7b-0cd7b515a051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:45.846649 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 985/1949] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:45.859821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:45.861581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:45.934836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f8dd7-e7b8-4db5-81c1-1ee58bc96804 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:45.943036 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3bcb8-501e-4b20-b5a9-832987f5910d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:45.944468 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3bcb8-501e-4b20-b5a9-832987f5910d service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:45.944783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3bcb8-501e-4b20-b5a9-832987f5910d service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:45.945091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3bcb8-501e-4b20-b5a9-832987f5910d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:45.960783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3bcb8-501e-4b20-b5a9-832987f5910d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:45.963136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3bcb8-501e-4b20-b5a9-832987f5910d service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:46.000985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 965/1950] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:46.343563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956fcb75-fafd-428d-a7ae-79f2f8ef29ee None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:46.344011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956fcb75-fafd-428d-a7ae-79f2f8ef29ee None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:46.344105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956fcb75-fafd-428d-a7ae-79f2f8ef29ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:46.346372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3bcb8-501e-4b20-b5a9-832987f5910d service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:46.347883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-64b3bcb8-501e-4b20-b5a9-832987f5910d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:46.366906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3bcb8-501e-4b20-b5a9-832987f5910d service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:46.371733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:46.372494 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:46.372849 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:46.373165 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] PATH_INFO: `/v3/projects/32f9b858351443e7ac78481cb57ce46f` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:46.374862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=32f9b858351443e7ac78481cb57ce46f)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:46.375252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-956fcb75-fafd-428d-a7ae-79f2f8ef29ee None None] MFA Rules not processed for user `bf0c0131372345e6b23260b466ec8ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:46.376542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-956fcb75-fafd-428d-a7ae-79f2f8ef29ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:46.376815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:46.434408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32f9b858351443e7ac78481cb57ce46f', 'request_id': 'req-e35709cd-3ee1-454f-ab27-442eb2632acf'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:46.445854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32f9b858351443e7ac78481cb57ce46f', 'request_id': 'req-e35709cd-3ee1-454f-ab27-442eb2632acf'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:46.453579 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32f9b858351443e7ac78481cb57ce46f', 'request_id': 'req-e35709cd-3ee1-454f-ab27-442eb2632acf'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:46.460342 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 986/1951] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:46.471975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956fcb75-fafd-428d-a7ae-79f2f8ef29ee None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:46.472257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] The token cache is being invalidate because project 32f9b858351443e7ac78481cb57ce46f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:46.472872 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:46.473274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:46.474504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-956fcb75-fafd-428d-a7ae-79f2f8ef29ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:46.474836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 966/1952] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:46 2022] DELETE /identity/v3/projects/32f9b858351443e7ac78481cb57ce46f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:46.481480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:46.483746 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:46.496813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:46.501568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:46.561854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956fcb75-fafd-428d-a7ae-79f2f8ef29ee None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:46.576474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35709cd-3ee1-454f-ab27-442eb2632acf admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:46.581730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:46.582612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:46.582893 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:46.583285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] PATH_INFO: `/v3/users/50a95c5f628f4835af28afce075fa97d` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:46.584982 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50a95c5f628f4835af28afce075fa97d)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:46.587021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:46.615789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45378177-7c00-48f0-b45e-d36eb4e70659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:46.616534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378177-7c00-48f0-b45e-d36eb4e70659 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:46.617092 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378177-7c00-48f0-b45e-d36eb4e70659 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:46.617424 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378177-7c00-48f0-b45e-d36eb4e70659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:46.619677 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-45378177-7c00-48f0-b45e-d36eb4e70659 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:46.643247 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50a95c5f628f4835af28afce075fa97d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:46.653903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50a95c5f628f4835af28afce075fa97d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:46.669126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50a95c5f628f4835af28afce075fa97d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:46.676657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50a95c5f628f4835af28afce075fa97d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:46.677384 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50a95c5f628f4835af28afce075fa97d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:46.683748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 967/1953] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:46 2022] DELETE /identity/v3/users/50a95c5f628f4835af28afce075fa97d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:46.695911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45378177-7c00-48f0-b45e-d36eb4e70659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:46.697610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45378177-7c00-48f0-b45e-d36eb4e70659 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:46.709885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 987/1954] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:47.244140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:47.249089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:47.405127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6204e02-6b3b-45af-9d8e-4c6750a875c4 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:47.412326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05564ad-9629-4c84-827d-94864acdbc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:47.413264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05564ad-9629-4c84-827d-94864acdbc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:47.413709 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05564ad-9629-4c84-827d-94864acdbc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:47.414170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05564ad-9629-4c84-827d-94864acdbc96 admin admin] PATH_INFO: `/v3/projects/a089850e154e4df79765546bf55a3d52` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:47.415559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05564ad-9629-4c84-827d-94864acdbc96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a089850e154e4df79765546bf55a3d52)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:47.417688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05564ad-9629-4c84-827d-94864acdbc96 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:47.460906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:47.474304 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45378177-7c00-48f0-b45e-d36eb4e70659 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:47.476291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-45378177-7c00-48f0-b45e-d36eb4e70659 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:47.476420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-e05564ad-9629-4c84-827d-94864acdbc96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a089850e154e4df79765546bf55a3d52', 'request_id': 'req-e05564ad-9629-4c84-827d-94864acdbc96'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:47.486651 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-e05564ad-9629-4c84-827d-94864acdbc96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a089850e154e4df79765546bf55a3d52', 'request_id': 'req-e05564ad-9629-4c84-827d-94864acdbc96'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:47.493551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45378177-7c00-48f0-b45e-d36eb4e70659 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:47.494672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-e05564ad-9629-4c84-827d-94864acdbc96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a089850e154e4df79765546bf55a3d52', 'request_id': 'req-e05564ad-9629-4c84-827d-94864acdbc96'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:47.508891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-e05564ad-9629-4c84-827d-94864acdbc96 admin admin] The token cache is being invalidate because project a089850e154e4df79765546bf55a3d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:47.509301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-e05564ad-9629-4c84-827d-94864acdbc96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:47.510226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-e05564ad-9629-4c84-827d-94864acdbc96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:47.517264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 968/1955] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:47 2022] DELETE /identity/v3/projects/a089850e154e4df79765546bf55a3d52 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:47.521823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d21dc7-5d69-4d8c-bd64-f1bd2bdab7f0 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:47.522671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d21dc7-5d69-4d8c-bd64-f1bd2bdab7f0 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:47.522671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d21dc7-5d69-4d8c-bd64-f1bd2bdab7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:47.545533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-22d21dc7-5d69-4d8c-bd64-f1bd2bdab7f0 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:47.546858 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-22d21dc7-5d69-4d8c-bd64-f1bd2bdab7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:47.564842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ee13d-a5c4-46a4-a21e-971ec43a8b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:47.566032 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ee13d-a5c4-46a4-a21e-971ec43a8b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:47.567076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ee13d-a5c4-46a4-a21e-971ec43a8b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:47.568052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ee13d-a5c4-46a4-a21e-971ec43a8b8f admin admin] PATH_INFO: `/v3/users/fc40b02de0bf43e08070e79440d55480` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:47.571371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ee13d-a5c4-46a4-a21e-971ec43a8b8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc40b02de0bf43e08070e79440d55480)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:47.578108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ee13d-a5c4-46a4-a21e-971ec43a8b8f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:47.640524 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:47.680210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 969/1956] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:47.690366 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3252d7-b93c-40d0-b512-8c840da9104e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:47.692747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3252d7-b93c-40d0-b512-8c840da9104e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:47.692966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-614ee13d-a5c4-46a4-a21e-971ec43a8b8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc40b02de0bf43e08070e79440d55480'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:47.693502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3252d7-b93c-40d0-b512-8c840da9104e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:47.693944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-614ee13d-a5c4-46a4-a21e-971ec43a8b8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc40b02de0bf43e08070e79440d55480'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:47.707889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-614ee13d-a5c4-46a4-a21e-971ec43a8b8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc40b02de0bf43e08070e79440d55480'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:47.719103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-2a3252d7-b93c-40d0-b512-8c840da9104e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:47.720326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2a3252d7-b93c-40d0-b512-8c840da9104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:47.724339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-614ee13d-a5c4-46a4-a21e-971ec43a8b8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc40b02de0bf43e08070e79440d55480'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:47.728702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 970/1957] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:47.733330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3252d7-b93c-40d0-b512-8c840da9104e None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:47.735703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-614ee13d-a5c4-46a4-a21e-971ec43a8b8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc40b02de0bf43e08070e79440d55480'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:47.745492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3252d7-b93c-40d0-b512-8c840da9104e None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:47.746737 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 988/1958] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:47 2022] DELETE /identity/v3/users/fc40b02de0bf43e08070e79440d55480 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:47.748452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a366a-032e-4a2b-875c-f51b567b8e2c None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:47.748943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a366a-032e-4a2b-875c-f51b567b8e2c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:47.749236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a366a-032e-4a2b-875c-f51b567b8e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:47.775251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-e60a366a-032e-4a2b-875c-f51b567b8e2c None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:47.776856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e60a366a-032e-4a2b-875c-f51b567b8e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:47.813456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3eb31-6fa0-4697-bd4a-fae6f3d9fe56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:47.814213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3eb31-6fa0-4697-bd4a-fae6f3d9fe56 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:47.814506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3eb31-6fa0-4697-bd4a-fae6f3d9fe56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:47.814790 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3eb31-6fa0-4697-bd4a-fae6f3d9fe56 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:47.815570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3eb31-6fa0-4697-bd4a-fae6f3d9fe56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:47.815882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3eb31-6fa0-4697-bd4a-fae6f3d9fe56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:47.817535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3eb31-6fa0-4697-bd4a-fae6f3d9fe56 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:47.830927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 971/1959] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:46:47.835325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3eb31-6fa0-4697-bd4a-fae6f3d9fe56 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:47.837104 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-bcc3eb31-6fa0-4697-bd4a-fae6f3d9fe56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:47.849294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3eb31-6fa0-4697-bd4a-fae6f3d9fe56 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:47.853240 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 989/1960] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:47.854221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a4f89-83ba-4de8-8ee2-47888e964ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:47.854999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a4f89-83ba-4de8-8ee2-47888e964ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:47.855363 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a4f89-83ba-4de8-8ee2-47888e964ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:47.855719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a4f89-83ba-4de8-8ee2-47888e964ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:47.856512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a4f89-83ba-4de8-8ee2-47888e964ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:47.859031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a4f89-83ba-4de8-8ee2-47888e964ec5 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:47.863399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a366a-032e-4a2b-875c-f51b567b8e2c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:47.864800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e60a366a-032e-4a2b-875c-f51b567b8e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:47.893512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 972/1961] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:47 2022] POST /identity/v3/projects => generated 409 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:47.895460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a4f89-83ba-4de8-8ee2-47888e964ec5 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:47.929136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a366a-032e-4a2b-875c-f51b567b8e2c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:47.934927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:47.977042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a4f89-83ba-4de8-8ee2-47888e964ec5 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:47.981901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ed53e-b6eb-4cfc-abdb-8456d4f30e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:47.983161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ed53e-b6eb-4cfc-abdb-8456d4f30e2e admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:47.983513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ed53e-b6eb-4cfc-abdb-8456d4f30e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:47.983931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ed53e-b6eb-4cfc-abdb-8456d4f30e2e admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:47.984673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ed53e-b6eb-4cfc-abdb-8456d4f30e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:47.986662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ed53e-b6eb-4cfc-abdb-8456d4f30e2e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:47.988135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73edf7b-0412-4d66-9fbb-2bfea9ff5073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:47.988950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73edf7b-0412-4d66-9fbb-2bfea9ff5073 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:47.989361 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73edf7b-0412-4d66-9fbb-2bfea9ff5073 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:47.989742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73edf7b-0412-4d66-9fbb-2bfea9ff5073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.001724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73edf7b-0412-4d66-9fbb-2bfea9ff5073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.003606 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73edf7b-0412-4d66-9fbb-2bfea9ff5073 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.013304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 973/1962] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:47 2022] POST /identity/v3/users => generated 305 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:48.014581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 990/1963] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:46:48.017603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ed53e-b6eb-4cfc-abdb-8456d4f30e2e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:48.029687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ed53e-b6eb-4cfc-abdb-8456d4f30e2e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:48.038100 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3657b6-fbe3-447e-8c8f-f8f1d5012c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:48.038961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3657b6-fbe3-447e-8c8f-f8f1d5012c9e admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.039342 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3657b6-fbe3-447e-8c8f-f8f1d5012c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.039519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3657b6-fbe3-447e-8c8f-f8f1d5012c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.040753 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3657b6-fbe3-447e-8c8f-f8f1d5012c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.042376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3657b6-fbe3-447e-8c8f-f8f1d5012c9e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.055091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 974/1964] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:48.057815 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73edf7b-0412-4d66-9fbb-2bfea9ff5073 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:48.059312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3657b6-fbe3-447e-8c8f-f8f1d5012c9e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:48.059964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d73edf7b-0412-4d66-9fbb-2bfea9ff5073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:48.070180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3657b6-fbe3-447e-8c8f-f8f1d5012c9e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:48.073114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:48.076365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b7d5c-2578-4cbe-ad25-622faa82759e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:48.077501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b7d5c-2578-4cbe-ad25-622faa82759e admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.077850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b7d5c-2578-4cbe-ad25-622faa82759e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.078446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b7d5c-2578-4cbe-ad25-622faa82759e admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.079684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b7d5c-2578-4cbe-ad25-622faa82759e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.079804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73edf7b-0412-4d66-9fbb-2bfea9ff5073 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:48.081727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b7d5c-2578-4cbe-ad25-622faa82759e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.093091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c585662-310f-4c43-b8e4-27d1959e5155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:48.093711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c585662-310f-4c43-b8e4-27d1959e5155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.095578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c585662-310f-4c43-b8e4-27d1959e5155 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.095855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c585662-310f-4c43-b8e4-27d1959e5155 admin admin] PATH_INFO: `/v3/users/cbc551ed018b402e83b460933698c593` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.097376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c585662-310f-4c43-b8e4-27d1959e5155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc551ed018b402e83b460933698c593)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.099424 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c585662-310f-4c43-b8e4-27d1959e5155 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.100222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 975/1965] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:48.101304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b7d5c-2578-4cbe-ad25-622faa82759e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:48.118624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b7d5c-2578-4cbe-ad25-622faa82759e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:48.133382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab18173-e010-4436-88aa-a78932180b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:48.134298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab18173-e010-4436-88aa-a78932180b89 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.134587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab18173-e010-4436-88aa-a78932180b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.135120 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab18173-e010-4436-88aa-a78932180b89 admin admin] PATH_INFO: `/v3/projects/1c1987ef62cd4f9184b719cd827df24b/users/10953286aa6e4c88b59572f8e6eadbd1/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.166949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab18173-e010-4436-88aa-a78932180b89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c1987ef62cd4f9184b719cd827df24b, user_id=10953286aa6e4c88b59572f8e6eadbd1, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.170701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab18173-e010-4436-88aa-a78932180b89 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.189983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 976/1966] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:48 2022] PUT /identity/v3/projects/1c1987ef62cd4f9184b719cd827df24b/users/10953286aa6e4c88b59572f8e6eadbd1/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:48.201235 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1c585662-310f-4c43-b8e4-27d1959e5155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc551ed018b402e83b460933698c593'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.202327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1c585662-310f-4c43-b8e4-27d1959e5155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc551ed018b402e83b460933698c593'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.205248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28506b-ac3f-46fc-a768-61a41ed3ea3f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.205562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28506b-ac3f-46fc-a768-61a41ed3ea3f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.206024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28506b-ac3f-46fc-a768-61a41ed3ea3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.212986 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1c585662-310f-4c43-b8e4-27d1959e5155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc551ed018b402e83b460933698c593'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.230916 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1c585662-310f-4c43-b8e4-27d1959e5155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc551ed018b402e83b460933698c593'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.239042 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1c585662-310f-4c43-b8e4-27d1959e5155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc551ed018b402e83b460933698c593'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.244880 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3f28506b-ac3f-46fc-a768-61a41ed3ea3f None None] MFA Rules not processed for user `10953286aa6e4c88b59572f8e6eadbd1`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:48.246160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 991/1967] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:48 2022] DELETE /identity/v3/users/cbc551ed018b402e83b460933698c593 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:48.246291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3f28506b-ac3f-46fc-a768-61a41ed3ea3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:48.250556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd0513-1654-4576-a55f-0a443ae514b1 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.251366 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd0513-1654-4576-a55f-0a443ae514b1 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.251366 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd0513-1654-4576-a55f-0a443ae514b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.272811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c3bd0513-1654-4576-a55f-0a443ae514b1 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:48.274371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c3bd0513-1654-4576-a55f-0a443ae514b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:48.378973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 977/1968] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:48 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:48.383332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28506b-ac3f-46fc-a768-61a41ed3ea3f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:48.385111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3f28506b-ac3f-46fc-a768-61a41ed3ea3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:48.401373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28506b-ac3f-46fc-a768-61a41ed3ea3f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:48.407872 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 992/1969] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:48.410783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011abb2-5220-45a1-8283-ef96e6468f01 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.411480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011abb2-5220-45a1-8283-ef96e6468f01 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.412810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecec93-260b-45ea-84f5-cf920dc0e321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:48.413511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecec93-260b-45ea-84f5-cf920dc0e321 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.413796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecec93-260b-45ea-84f5-cf920dc0e321 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.414057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecec93-260b-45ea-84f5-cf920dc0e321 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.414780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ecec93-260b-45ea-84f5-cf920dc0e321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.416757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ecec93-260b-45ea-84f5-cf920dc0e321 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.419475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011abb2-5220-45a1-8283-ef96e6468f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.442388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a011abb2-5220-45a1-8283-ef96e6468f01 None None] MFA Rules not processed for user `bf0c0131372345e6b23260b466ec8ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:48.443896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a011abb2-5220-45a1-8283-ef96e6468f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:48.448621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 978/1970] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:48 2022] POST /identity/v3/projects => generated 409 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:48.449921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecec93-260b-45ea-84f5-cf920dc0e321 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:48.452423 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-99ecec93-260b-45ea-84f5-cf920dc0e321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:48.509245 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:48.522393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 993/1971] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:48.527252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011abb2-5220-45a1-8283-ef96e6468f01 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:48.534310 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecec93-260b-45ea-84f5-cf920dc0e321 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:48.537776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011abb2-5220-45a1-8283-ef96e6468f01 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:48.541273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65797765-1488-4ded-91d2-a31a0fe9e092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:48.545448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65797765-1488-4ded-91d2-a31a0fe9e092 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.546962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65797765-1488-4ded-91d2-a31a0fe9e092 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.547557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65797765-1488-4ded-91d2-a31a0fe9e092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.565907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:48.629480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa46940-c910-4ae0-bfac-cf15faade575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:48.630702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa46940-c910-4ae0-bfac-cf15faade575 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.631092 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa46940-c910-4ae0-bfac-cf15faade575 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.631428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa46940-c910-4ae0-bfac-cf15faade575 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.632330 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa46940-c910-4ae0-bfac-cf15faade575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.634659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa46940-c910-4ae0-bfac-cf15faade575 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.652482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65797765-1488-4ded-91d2-a31a0fe9e092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.655171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65797765-1488-4ded-91d2-a31a0fe9e092 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.664964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 994/1972] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:48 2022] POST /identity/v3/users => generated 311 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:48.666137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa46940-c910-4ae0-bfac-cf15faade575 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:48.667114 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 979/1973] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:48.668371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0fa46940-c910-4ae0-bfac-cf15faade575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:48.668986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65797765-1488-4ded-91d2-a31a0fe9e092 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:48.680335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65797765-1488-4ded-91d2-a31a0fe9e092 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:48.681528 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa46940-c910-4ae0-bfac-cf15faade575 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:48.686719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29933c-017a-45c3-bc12-f468bbf0ed7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:48.687276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:48.688158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.688431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29933c-017a-45c3-bc12-f468bbf0ed7e admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.688563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.688749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] PATH_INFO: `/v3/projects/13ff188603624645a8a7032bb93b21f4` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.689032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29933c-017a-45c3-bc12-f468bbf0ed7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.689464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29933c-017a-45c3-bc12-f468bbf0ed7e admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.690039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=13ff188603624645a8a7032bb93b21f4)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.691976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.698311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29933c-017a-45c3-bc12-f468bbf0ed7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.702312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29933c-017a-45c3-bc12-f468bbf0ed7e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.718038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 980/1974] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:48.724908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29933c-017a-45c3-bc12-f468bbf0ed7e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:48.736773 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29933c-017a-45c3-bc12-f468bbf0ed7e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:48.768705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13ff188603624645a8a7032bb93b21f4', 'request_id': 'req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.782150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13ff188603624645a8a7032bb93b21f4', 'request_id': 'req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.793751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13ff188603624645a8a7032bb93b21f4', 'request_id': 'req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.813709 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] The token cache is being invalidate because project 13ff188603624645a8a7032bb93b21f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:48.814086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.814916 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.816758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 995/1975] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:48 2022] DELETE /identity/v3/projects/13ff188603624645a8a7032bb93b21f4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:48.821222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c2e92-b624-4c01-879c-f53b0d933492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:48.821478 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:48.822139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c2e92-b624-4c01-879c-f53b0d933492 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.822423 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c2e92-b624-4c01-879c-f53b0d933492 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.822697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c2e92-b624-4c01-879c-f53b0d933492 admin admin] PATH_INFO: `/v3/projects/bf5cb4a891a74bf486c3b0e5211f8fca/users/a1fe2b0b143c45278ad5891300f83782/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.823417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:48.844167 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71a6e8-e57e-4339-af02-da2c49b8bc4c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:48.850819 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:48.852241 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.852685 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.853134 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] PATH_INFO: `/v3/users/ddfb390192de436ab832a184fe4fa455` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.854788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddfb390192de436ab832a184fe4fa455)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.857238 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.857769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c2e92-b624-4c01-879c-f53b0d933492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf5cb4a891a74bf486c3b0e5211f8fca, user_id=a1fe2b0b143c45278ad5891300f83782, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.860560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c2e92-b624-4c01-879c-f53b0d933492 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.878099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 981/1976] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:48 2022] PUT /identity/v3/projects/bf5cb4a891a74bf486c3b0e5211f8fca/users/a1fe2b0b143c45278ad5891300f83782/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:48.881820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c2e92-b624-4c01-879c-f53b0d933492 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:48.883750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c53c2e92-b624-4c01-879c-f53b0d933492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:48.928886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddfb390192de436ab832a184fe4fa455'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.929769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddfb390192de436ab832a184fe4fa455'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.944292 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddfb390192de436ab832a184fe4fa455'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.964412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddfb390192de436ab832a184fe4fa455'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.975578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddfb390192de436ab832a184fe4fa455'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:48.984305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c2e92-b624-4c01-879c-f53b0d933492 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:48.989589 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 996/1977] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:48 2022] DELETE /identity/v3/users/ddfb390192de436ab832a184fe4fa455 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:48.993031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed8d19-3d39-424e-9562-ba6781fd6d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:48.993031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed8d19-3d39-424e-9562-ba6781fd6d70 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:48.993031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed8d19-3d39-424e-9562-ba6781fd6d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:48.993031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed8d19-3d39-424e-9562-ba6781fd6d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:48.993505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed8d19-3d39-424e-9562-ba6781fd6d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:48.994581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed8d19-3d39-424e-9562-ba6781fd6d70 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:48.997712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:49.007788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 982/1978] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:49.012108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:49.013838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:49.026583 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=78,keystone:INSERT=5,keystone:DELETE=17 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:49.093153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed8d19-3d39-424e-9562-ba6781fd6d70 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:49.094965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-25ed8d19-3d39-424e-9562-ba6781fd6d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:49.115562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed8d19-3d39-424e-9562-ba6781fd6d70 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:49.132680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:49.132680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:49.132680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:49.132680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] PATH_INFO: `/v3/projects/7eb4a3a2f95046d3be0b8f4abe69f9b8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:49.134486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eb4a3a2f95046d3be0b8f4abe69f9b8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:49.137146 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:49.138154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915ead0-4c47-4022-9e93-3842f44b5a89 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:49.165723 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7ad2e-3d3d-473e-82e1-db2dfc01913e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:49.166970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7ad2e-3d3d-473e-82e1-db2dfc01913e admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:49.167326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7ad2e-3d3d-473e-82e1-db2dfc01913e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:49.167613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7ad2e-3d3d-473e-82e1-db2dfc01913e admin admin] PATH_INFO: `/v3/projects/bf5cb4a891a74bf486c3b0e5211f8fca/users/a1fe2b0b143c45278ad5891300f83782/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:49.171442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7ad2e-3d3d-473e-82e1-db2dfc01913e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf5cb4a891a74bf486c3b0e5211f8fca, user_id=a1fe2b0b143c45278ad5891300f83782, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:49.174851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7ad2e-3d3d-473e-82e1-db2dfc01913e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:49.203276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 997/1979] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:49 2022] PUT /identity/v3/projects/bf5cb4a891a74bf486c3b0e5211f8fca/users/a1fe2b0b143c45278ad5891300f83782/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:49.215009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cae95-6c81-4721-81fc-083b033f6260 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:49.215889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cae95-6c81-4721-81fc-083b033f6260 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:49.216905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eb4a3a2f95046d3be0b8f4abe69f9b8', 'request_id': 'req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.217417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cae95-6c81-4721-81fc-083b033f6260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:49.228511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eb4a3a2f95046d3be0b8f4abe69f9b8', 'request_id': 'req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.237866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eb4a3a2f95046d3be0b8f4abe69f9b8', 'request_id': 'req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.257747 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-e99cae95-6c81-4721-81fc-083b033f6260 None None] MFA Rules not processed for user `a1fe2b0b143c45278ad5891300f83782`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:49.260447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e99cae95-6c81-4721-81fc-083b033f6260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:49.274766 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] The token cache is being invalidate because project 7eb4a3a2f95046d3be0b8f4abe69f9b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:49.275814 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.276743 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.278777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 983/1980] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:49 2022] DELETE /identity/v3/projects/7eb4a3a2f95046d3be0b8f4abe69f9b8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:49.282804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:49.288609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:49.389645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 998/1981] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:49 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:49.397522 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cae95-6c81-4721-81fc-083b033f6260 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:49.400040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e99cae95-6c81-4721-81fc-083b033f6260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:49.414770 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae7e56-4d90-4e9c-8ff3-8350e8848a19 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:49.423515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb61038-a019-455e-92a6-94151803b3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:49.425170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb61038-a019-455e-92a6-94151803b3d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:49.425582 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb61038-a019-455e-92a6-94151803b3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:49.425945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb61038-a019-455e-92a6-94151803b3d0 admin admin] PATH_INFO: `/v3/users/ded1cd942879421f9d9c93bbf8f7d151` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:49.440607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb61038-a019-455e-92a6-94151803b3d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded1cd942879421f9d9c93bbf8f7d151)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:49.443295 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb61038-a019-455e-92a6-94151803b3d0 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:49.512138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cae95-6c81-4721-81fc-083b033f6260 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:49.518373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddb7be-01b0-42c5-8255-b70f1571f4ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:49.519035 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddb7be-01b0-42c5-8255-b70f1571f4ab service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:49.519397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddb7be-01b0-42c5-8255-b70f1571f4ab service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:49.519679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddb7be-01b0-42c5-8255-b70f1571f4ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:49.532288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddb7be-01b0-42c5-8255-b70f1571f4ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:49.534138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddb7be-01b0-42c5-8255-b70f1571f4ab service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:49.556360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 999/1982] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:49 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:46:49.556530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-adb61038-a019-455e-92a6-94151803b3d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded1cd942879421f9d9c93bbf8f7d151'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.557362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddb7be-01b0-42c5-8255-b70f1571f4ab service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:49.562692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-95ddb7be-01b0-42c5-8255-b70f1571f4ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:49.570241 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-adb61038-a019-455e-92a6-94151803b3d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded1cd942879421f9d9c93bbf8f7d151'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.597520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddb7be-01b0-42c5-8255-b70f1571f4ab service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:49.604098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-adb61038-a019-455e-92a6-94151803b3d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded1cd942879421f9d9c93bbf8f7d151'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.609617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:49.612170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:49.612738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:49.613612 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] PATH_INFO: `/v3/projects/d6d92b89f2ce4c189004c84a5b55811b` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:49.613769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-adb61038-a019-455e-92a6-94151803b3d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded1cd942879421f9d9c93bbf8f7d151'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.614781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-adb61038-a019-455e-92a6-94151803b3d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded1cd942879421f9d9c93bbf8f7d151'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.615500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6d92b89f2ce4c189004c84a5b55811b)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:49.617900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:49.619316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:49.623194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 984/1983] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:49 2022] DELETE /identity/v3/users/ded1cd942879421f9d9c93bbf8f7d151 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:49.672540 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:49.719575 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6d92b89f2ce4c189004c84a5b55811b', 'request_id': 'req-57809c9b-192e-4bca-8a55-c1dd9ab10538'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.733334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6d92b89f2ce4c189004c84a5b55811b', 'request_id': 'req-57809c9b-192e-4bca-8a55-c1dd9ab10538'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.745064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6d92b89f2ce4c189004c84a5b55811b', 'request_id': 'req-57809c9b-192e-4bca-8a55-c1dd9ab10538'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.767608 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] The token cache is being invalidate because project d6d92b89f2ce4c189004c84a5b55811b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:49.768064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.780140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:49.782659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1000/1984] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:49 2022] DELETE /identity/v3/projects/d6d92b89f2ce4c189004c84a5b55811b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:49.788150 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5b4ac-6fb6-4c07-80bc-4789c957c854 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:49.788562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5b4ac-6fb6-4c07-80bc-4789c957c854 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:49.789133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5b4ac-6fb6-4c07-80bc-4789c957c854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:49.795471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:49.797442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:49.829609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-cdf5b4ac-6fb6-4c07-80bc-4789c957c854 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:49.831457 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-cdf5b4ac-6fb6-4c07-80bc-4789c957c854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:49.945453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 985/1985] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:49.959998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615746c8-2e50-41c8-9c7c-9eca72511b47 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:49.961896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615746c8-2e50-41c8-9c7c-9eca72511b47 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:49.966203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615746c8-2e50-41c8-9c7c-9eca72511b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:49.968343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57809c9b-192e-4bca-8a55-c1dd9ab10538 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:49.972191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c370f-389b-44bf-aa71-4ba7212de326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:49.973044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c370f-389b-44bf-aa71-4ba7212de326 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:49.974072 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c370f-389b-44bf-aa71-4ba7212de326 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:49.974318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c370f-389b-44bf-aa71-4ba7212de326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:49.976356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2a5c370f-389b-44bf-aa71-4ba7212de326 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:50.022084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-615746c8-2e50-41c8-9c7c-9eca72511b47 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:50.025905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-615746c8-2e50-41c8-9c7c-9eca72511b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:50.043888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 986/1986] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:50.048803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615746c8-2e50-41c8-9c7c-9eca72511b47 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:50.067698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615746c8-2e50-41c8-9c7c-9eca72511b47 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:50.083050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5befb-3ef1-44ad-b4db-95f0ad4b7b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:50.084094 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5befb-3ef1-44ad-b4db-95f0ad4b7b8e admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:50.085667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5befb-3ef1-44ad-b4db-95f0ad4b7b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:50.086073 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5befb-3ef1-44ad-b4db-95f0ad4b7b8e admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:50.086850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5befb-3ef1-44ad-b4db-95f0ad4b7b8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:50.089780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5befb-3ef1-44ad-b4db-95f0ad4b7b8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:50.091530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5befb-3ef1-44ad-b4db-95f0ad4b7b8e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:50.102916 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c370f-389b-44bf-aa71-4ba7212de326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:50.107885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c370f-389b-44bf-aa71-4ba7212de326 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:50.130276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 987/1987] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:46:50.130889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1001/1988] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:49 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:50.134669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c370f-389b-44bf-aa71-4ba7212de326 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:50.159216 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c370f-389b-44bf-aa71-4ba7212de326 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:50.170415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ccb5d-6f2b-443a-b67f-53af1328b830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:50.172867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ccb5d-6f2b-443a-b67f-53af1328b830 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:50.175130 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ccb5d-6f2b-443a-b67f-53af1328b830 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:50.175444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ccb5d-6f2b-443a-b67f-53af1328b830 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:50.176394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ccb5d-6f2b-443a-b67f-53af1328b830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:50.178993 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ccb5d-6f2b-443a-b67f-53af1328b830 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:50.236232 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5befb-3ef1-44ad-b4db-95f0ad4b7b8e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:50.238707 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-94a5befb-3ef1-44ad-b4db-95f0ad4b7b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:50.257081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5befb-3ef1-44ad-b4db-95f0ad4b7b8e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:50.257421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1002/1989] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:50 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:50.258114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ccb5d-6f2b-443a-b67f-53af1328b830 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:50.370676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:50.390464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e92a8-f838-40c7-8091-74468b2246f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:50.391306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e92a8-f838-40c7-8091-74468b2246f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:50.391587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e92a8-f838-40c7-8091-74468b2246f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:50.391861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e92a8-f838-40c7-8091-74468b2246f2 admin admin] PATH_INFO: `/v3/projects/6a4b3b6dd4834965adf25e3a7a4f0076` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:50.397811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e92a8-f838-40c7-8091-74468b2246f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4b3b6dd4834965adf25e3a7a4f0076)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:50.403187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e92a8-f838-40c7-8091-74468b2246f2 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:50.418342 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ccb5d-6f2b-443a-b67f-53af1328b830 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:50.432264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52eb0ee-a1a5-4837-8b69-71e86597c8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:50.433004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52eb0ee-a1a5-4837-8b69-71e86597c8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:50.433214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52eb0ee-a1a5-4837-8b69-71e86597c8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:50.433495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52eb0ee-a1a5-4837-8b69-71e86597c8e8 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:50.434205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52eb0ee-a1a5-4837-8b69-71e86597c8e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:50.439885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52eb0ee-a1a5-4837-8b69-71e86597c8e8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:50.488939 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-cd6e92a8-f838-40c7-8091-74468b2246f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4b3b6dd4834965adf25e3a7a4f0076', 'request_id': 'req-cd6e92a8-f838-40c7-8091-74468b2246f2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:50.496465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1003/1990] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:50 2022] POST /identity/v3/users => generated 287 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:50.497658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52eb0ee-a1a5-4837-8b69-71e86597c8e8 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:50.502653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-cd6e92a8-f838-40c7-8091-74468b2246f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4b3b6dd4834965adf25e3a7a4f0076', 'request_id': 'req-cd6e92a8-f838-40c7-8091-74468b2246f2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:50.514751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-cd6e92a8-f838-40c7-8091-74468b2246f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4b3b6dd4834965adf25e3a7a4f0076', 'request_id': 'req-cd6e92a8-f838-40c7-8091-74468b2246f2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:50.516608 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52eb0ee-a1a5-4837-8b69-71e86597c8e8 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:50.544218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-cd6e92a8-f838-40c7-8091-74468b2246f2 admin admin] The token cache is being invalidate because project 6a4b3b6dd4834965adf25e3a7a4f0076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:50.544793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-cd6e92a8-f838-40c7-8091-74468b2246f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:50.554462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-cd6e92a8-f838-40c7-8091-74468b2246f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:50.562808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 988/1991] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:50 2022] DELETE /identity/v3/projects/6a4b3b6dd4834965adf25e3a7a4f0076 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:50.565658 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020e08b-7f0f-4d40-9cd7-16e984915a24 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:50.565985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020e08b-7f0f-4d40-9cd7-16e984915a24 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:50.566395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020e08b-7f0f-4d40-9cd7-16e984915a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:50.608945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-6020e08b-7f0f-4d40-9cd7-16e984915a24 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:50.610818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-6020e08b-7f0f-4d40-9cd7-16e984915a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:50.619216 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed95bb-1b32-4f3e-958e-1121f3945912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:50.620246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed95bb-1b32-4f3e-958e-1121f3945912 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:50.620426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed95bb-1b32-4f3e-958e-1121f3945912 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:50.620859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed95bb-1b32-4f3e-958e-1121f3945912 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:50.622139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed95bb-1b32-4f3e-958e-1121f3945912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:50.623722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed95bb-1b32-4f3e-958e-1121f3945912 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:50.639112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1004/1992] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:50.641583 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:50.643449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed95bb-1b32-4f3e-958e-1121f3945912 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:50.646053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-94ed95bb-1b32-4f3e-958e-1121f3945912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:50.662400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed95bb-1b32-4f3e-958e-1121f3945912 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:50.668368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b039a-9308-440c-9686-b784e1ddaac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:50.669142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b039a-9308-440c-9686-b784e1ddaac4 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:50.669923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b039a-9308-440c-9686-b784e1ddaac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:50.669923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b039a-9308-440c-9686-b784e1ddaac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:50.670462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b039a-9308-440c-9686-b784e1ddaac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:50.672383 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b039a-9308-440c-9686-b784e1ddaac4 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:50.715337 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1005/1993] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:50.721302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b039a-9308-440c-9686-b784e1ddaac4 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:50.758863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b039a-9308-440c-9686-b784e1ddaac4 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:50.776821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268112cb-c88b-425a-9912-0a19031ed2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:50.778153 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268112cb-c88b-425a-9912-0a19031ed2e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:50.778671 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268112cb-c88b-425a-9912-0a19031ed2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:50.779073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268112cb-c88b-425a-9912-0a19031ed2e4 admin admin] PATH_INFO: `/v3/projects/061203f8609740c2a80f6d6f10bb5421/users/9e60a97d2a1643d1b9c40deaee5b0c5c/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:50.799548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268112cb-c88b-425a-9912-0a19031ed2e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061203f8609740c2a80f6d6f10bb5421, user_id=9e60a97d2a1643d1b9c40deaee5b0c5c, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:50.802576 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268112cb-c88b-425a-9912-0a19031ed2e4 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:50.818214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 989/1994] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:50.824902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1006/1995] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:50 2022] PUT /identity/v3/projects/061203f8609740c2a80f6d6f10bb5421/users/9e60a97d2a1643d1b9c40deaee5b0c5c/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:50.827832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f94766-37bd-4607-8cfe-ce03da172975 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:50.828666 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f94766-37bd-4607-8cfe-ce03da172975 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:50.830115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f94766-37bd-4607-8cfe-ce03da172975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:50.836402 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12367594-5393-43de-af73-dd849463c47f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:50.837257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12367594-5393-43de-af73-dd849463c47f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:50.837257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12367594-5393-43de-af73-dd849463c47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:50.872133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-a0f94766-37bd-4607-8cfe-ce03da172975 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:50.872406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-12367594-5393-43de-af73-dd849463c47f None None] MFA Rules not processed for user `9e60a97d2a1643d1b9c40deaee5b0c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:50.875435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-12367594-5393-43de-af73-dd849463c47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:50.881491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a0f94766-37bd-4607-8cfe-ce03da172975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:50.963242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 990/1996] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:50.967756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1007/1997] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:50.968801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f94766-37bd-4607-8cfe-ce03da172975 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:50.972903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12367594-5393-43de-af73-dd849463c47f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:50.979403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f94766-37bd-4607-8cfe-ce03da172975 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:50.985640 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12367594-5393-43de-af73-dd849463c47f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:50.985833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e541a8f-f7da-4776-88c7-01cda222ceaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:50.986588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e541a8f-f7da-4776-88c7-01cda222ceaf admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:50.986764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e541a8f-f7da-4776-88c7-01cda222ceaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:50.987045 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e541a8f-f7da-4776-88c7-01cda222ceaf admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:50.987787 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e541a8f-f7da-4776-88c7-01cda222ceaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:50.988098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e541a8f-f7da-4776-88c7-01cda222ceaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:50.990070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e541a8f-f7da-4776-88c7-01cda222ceaf admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:50.992146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc402a40-38d4-4b3f-be21-b6c93dee51fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:50.992946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc402a40-38d4-4b3f-be21-b6c93dee51fa admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:50.993548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc402a40-38d4-4b3f-be21-b6c93dee51fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:50.993938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc402a40-38d4-4b3f-be21-b6c93dee51fa admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:50.995012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc402a40-38d4-4b3f-be21-b6c93dee51fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:50.997068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc402a40-38d4-4b3f-be21-b6c93dee51fa admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.009453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 991/1998] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:46:51.011238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e541a8f-f7da-4776-88c7-01cda222ceaf admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.031952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e541a8f-f7da-4776-88c7-01cda222ceaf admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.044222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7701bab-7c07-49b0-b537-a8b6c1e1f47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.046190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7701bab-7c07-49b0-b537-a8b6c1e1f47b admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.046479 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7701bab-7c07-49b0-b537-a8b6c1e1f47b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.047243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7701bab-7c07-49b0-b537-a8b6c1e1f47b admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.047959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7701bab-7c07-49b0-b537-a8b6c1e1f47b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.050450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1008/1999] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:50 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:51.052259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7701bab-7c07-49b0-b537-a8b6c1e1f47b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.054421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc402a40-38d4-4b3f-be21-b6c93dee51fa admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.062625 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:51.110834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 992/2000] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:51 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:51.117608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7701bab-7c07-49b0-b537-a8b6c1e1f47b admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.168509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc402a40-38d4-4b3f-be21-b6c93dee51fa admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.226063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7701bab-7c07-49b0-b537-a8b6c1e1f47b admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.241512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba400a80-63c9-4c04-9fd4-005ae8943753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.242283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba400a80-63c9-4c04-9fd4-005ae8943753 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.242574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba400a80-63c9-4c04-9fd4-005ae8943753 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.242853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba400a80-63c9-4c04-9fd4-005ae8943753 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.243566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba400a80-63c9-4c04-9fd4-005ae8943753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.248684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba400a80-63c9-4c04-9fd4-005ae8943753 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.311011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 993/2001] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:51 2022] POST /identity/v3/users => generated 284 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:51.311509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba400a80-63c9-4c04-9fd4-005ae8943753 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.313961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377f972-2c9e-4f77-b38d-d28572d3cb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.314779 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377f972-2c9e-4f77-b38d-d28572d3cb66 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.315076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377f972-2c9e-4f77-b38d-d28572d3cb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.315360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377f972-2c9e-4f77-b38d-d28572d3cb66 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.316347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377f972-2c9e-4f77-b38d-d28572d3cb66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.319569 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377f972-2c9e-4f77-b38d-d28572d3cb66 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.338298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba400a80-63c9-4c04-9fd4-005ae8943753 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.344236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eea8fd-8434-4ce2-8b61-1dfd8c264ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.345449 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eea8fd-8434-4ce2-8b61-1dfd8c264ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.345593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eea8fd-8434-4ce2-8b61-1dfd8c264ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.345796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eea8fd-8434-4ce2-8b61-1dfd8c264ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.346701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eea8fd-8434-4ce2-8b61-1dfd8c264ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.349124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eea8fd-8434-4ce2-8b61-1dfd8c264ba8 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.350756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:51.353483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1009/2002] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:51 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:51.358239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377f972-2c9e-4f77-b38d-d28572d3cb66 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.365279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 994/2003] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:51.369068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eea8fd-8434-4ce2-8b61-1dfd8c264ba8 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.372340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377f972-2c9e-4f77-b38d-d28572d3cb66 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.378437 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ea181-abd9-4aef-be46-e9d1e7425492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.379162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ea181-abd9-4aef-be46-e9d1e7425492 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.379471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ea181-abd9-4aef-be46-e9d1e7425492 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.379928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ea181-abd9-4aef-be46-e9d1e7425492 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.380836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ea181-abd9-4aef-be46-e9d1e7425492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.380961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eea8fd-8434-4ce2-8b61-1dfd8c264ba8 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.382622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ea181-abd9-4aef-be46-e9d1e7425492 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.386734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0f466-48c1-44f2-8db7-44bb3a89243a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.387513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0f466-48c1-44f2-8db7-44bb3a89243a admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.387916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0f466-48c1-44f2-8db7-44bb3a89243a admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.389092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0f466-48c1-44f2-8db7-44bb3a89243a admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.390022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0f466-48c1-44f2-8db7-44bb3a89243a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.391929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0f466-48c1-44f2-8db7-44bb3a89243a admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.396686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1010/2004] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:51.399364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ea181-abd9-4aef-be46-e9d1e7425492 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.408008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 995/2005] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:51.412102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0f466-48c1-44f2-8db7-44bb3a89243a admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.413759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ea181-abd9-4aef-be46-e9d1e7425492 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.419647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c81ada-5f28-4335-b580-71876c904551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.420753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c81ada-5f28-4335-b580-71876c904551 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.421140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c81ada-5f28-4335-b580-71876c904551 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.421520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c81ada-5f28-4335-b580-71876c904551 admin admin] PATH_INFO: `/v3/projects/0653ece80d9f4c0e943937b1f4bc9784/users/aa22756db4144f1cbdfe21ef8c7cca2d/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.426424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0f466-48c1-44f2-8db7-44bb3a89243a admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.430850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd2b2b-a403-4e9f-881a-48b87613cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.432095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd2b2b-a403-4e9f-881a-48b87613cd8c admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.432095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd2b2b-a403-4e9f-881a-48b87613cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.432314 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd2b2b-a403-4e9f-881a-48b87613cd8c admin admin] PATH_INFO: `/v3/projects/bb93e9f857fd4692b24ccf2b600ce511/users/f3c9b5d0309a4ecd9c724795b26c5174/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.440825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c81ada-5f28-4335-b580-71876c904551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0653ece80d9f4c0e943937b1f4bc9784, user_id=aa22756db4144f1cbdfe21ef8c7cca2d, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.443627 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c81ada-5f28-4335-b580-71876c904551 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.449867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd2b2b-a403-4e9f-881a-48b87613cd8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb93e9f857fd4692b24ccf2b600ce511, user_id=f3c9b5d0309a4ecd9c724795b26c5174, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.453520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd2b2b-a403-4e9f-881a-48b87613cd8c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.467616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1011/2006] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:51 2022] PUT /identity/v3/projects/0653ece80d9f4c0e943937b1f4bc9784/users/aa22756db4144f1cbdfe21ef8c7cca2d/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:51.468824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c81ada-5f28-4335-b580-71876c904551 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.470461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 996/2007] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:51 2022] PUT /identity/v3/projects/bb93e9f857fd4692b24ccf2b600ce511/users/f3c9b5d0309a4ecd9c724795b26c5174/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:51.482421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d7ac9-8732-4211-9b08-00c1d0bfcd97 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.482716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d7ac9-8732-4211-9b08-00c1d0bfcd97 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.483000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d7ac9-8732-4211-9b08-00c1d0bfcd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.513927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-602d7ac9-8732-4211-9b08-00c1d0bfcd97 None None] MFA Rules not processed for user `f3c9b5d0309a4ecd9c724795b26c5174`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:51.515166 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-602d7ac9-8732-4211-9b08-00c1d0bfcd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:51.548586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c81ada-5f28-4335-b580-71876c904551 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.558929 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:51.598587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 997/2008] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:51.602994 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d7ac9-8732-4211-9b08-00c1d0bfcd97 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.614775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5016af7-c677-4894-9758-59a84069f468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.615465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5016af7-c677-4894-9758-59a84069f468 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.615752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5016af7-c677-4894-9758-59a84069f468 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.616039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5016af7-c677-4894-9758-59a84069f468 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.616818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5016af7-c677-4894-9758-59a84069f468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.618425 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5016af7-c677-4894-9758-59a84069f468 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.631599 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1012/2009] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:51.632722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5016af7-c677-4894-9758-59a84069f468 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.645328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5016af7-c677-4894-9758-59a84069f468 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.651538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d646dd-0092-4ed1-af8d-ef9373e72d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.652493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d646dd-0092-4ed1-af8d-ef9373e72d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.652830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d646dd-0092-4ed1-af8d-ef9373e72d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.653286 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d646dd-0092-4ed1-af8d-ef9373e72d86 admin admin] PATH_INFO: `/v3/projects/0653ece80d9f4c0e943937b1f4bc9784/users/aa22756db4144f1cbdfe21ef8c7cca2d/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.656388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d646dd-0092-4ed1-af8d-ef9373e72d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0653ece80d9f4c0e943937b1f4bc9784, user_id=aa22756db4144f1cbdfe21ef8c7cca2d, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.659817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d646dd-0092-4ed1-af8d-ef9373e72d86 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.671219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1013/2010] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:51 2022] PUT /identity/v3/projects/0653ece80d9f4c0e943937b1f4bc9784/users/aa22756db4144f1cbdfe21ef8c7cca2d/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:51.681349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d7ac9-8732-4211-9b08-00c1d0bfcd97 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.681734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3e56f-6706-4c32-af5d-4f17a1e2cc7b None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.682040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3e56f-6706-4c32-af5d-4f17a1e2cc7b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.682331 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3e56f-6706-4c32-af5d-4f17a1e2cc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.719961 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-28b3e56f-6706-4c32-af5d-4f17a1e2cc7b None None] MFA Rules not processed for user `aa22756db4144f1cbdfe21ef8c7cca2d`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:51.721280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-28b3e56f-6706-4c32-af5d-4f17a1e2cc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:51.753964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:51.764815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b0642-8d34-4374-99bf-2cea09890930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.766216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b0642-8d34-4374-99bf-2cea09890930 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.766772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b0642-8d34-4374-99bf-2cea09890930 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.767492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b0642-8d34-4374-99bf-2cea09890930 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.768222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b0642-8d34-4374-99bf-2cea09890930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.770620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b0642-8d34-4374-99bf-2cea09890930 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.816755 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 998/2011] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:51 2022] POST /identity/v3/projects => generated 365 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:51.820882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b0642-8d34-4374-99bf-2cea09890930 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.825756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1014/2012] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:51.838231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3e56f-6706-4c32-af5d-4f17a1e2cc7b None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.846594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-28b3e56f-6706-4c32-af5d-4f17a1e2cc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:51.908526 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b0642-8d34-4374-99bf-2cea09890930 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.916077 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6a9b-9b0f-4197-94e5-49a355117250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.916959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6a9b-9b0f-4197-94e5-49a355117250 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.917346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6a9b-9b0f-4197-94e5-49a355117250 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.918161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6a9b-9b0f-4197-94e5-49a355117250 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.919503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a6a9b-9b0f-4197-94e5-49a355117250 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.923338 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a6a9b-9b0f-4197-94e5-49a355117250 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:51.959445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 999/2013] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:51 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:51.968534 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6a9b-9b0f-4197-94e5-49a355117250 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:51.979234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6a9b-9b0f-4197-94e5-49a355117250 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:51.985741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87667464-1367-4bd1-a160-8d80d8303361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:51.986432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87667464-1367-4bd1-a160-8d80d8303361 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:51.986720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87667464-1367-4bd1-a160-8d80d8303361 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:51.987107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87667464-1367-4bd1-a160-8d80d8303361 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:51.987881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87667464-1367-4bd1-a160-8d80d8303361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:51.989685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87667464-1367-4bd1-a160-8d80d8303361 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:52.013092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1000/2014] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:52.016137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87667464-1367-4bd1-a160-8d80d8303361 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:52.023029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3e56f-6706-4c32-af5d-4f17a1e2cc7b None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:52.034495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87667464-1367-4bd1-a160-8d80d8303361 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:52.042049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2c7eb-162c-4e84-b082-1d2665b21bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:52.043673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2c7eb-162c-4e84-b082-1d2665b21bfa admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:52.043673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2c7eb-162c-4e84-b082-1d2665b21bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:52.043878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2c7eb-162c-4e84-b082-1d2665b21bfa admin admin] PATH_INFO: `/v3/projects/c03684533b434d25b6b132c91e01505e/users/b7a346bbe3d14adeb18c55903aee642b/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:52.065256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2c7eb-162c-4e84-b082-1d2665b21bfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03684533b434d25b6b132c91e01505e, user_id=b7a346bbe3d14adeb18c55903aee642b, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:52.070254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2c7eb-162c-4e84-b082-1d2665b21bfa admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:52.076950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:52.094669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1001/2015] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:52 2022] PUT /identity/v3/projects/c03684533b434d25b6b132c91e01505e/users/b7a346bbe3d14adeb18c55903aee642b/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:52.101748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2c7eb-162c-4e84-b082-1d2665b21bfa admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:52.119379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c8d99-5bf6-4cb4-bf1e-1317c1143d17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:52.120113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c8d99-5bf6-4cb4-bf1e-1317c1143d17 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:52.120473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c8d99-5bf6-4cb4-bf1e-1317c1143d17 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:52.120744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c8d99-5bf6-4cb4-bf1e-1317c1143d17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:52.198528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2c7eb-162c-4e84-b082-1d2665b21bfa admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:52.205375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c8d99-5bf6-4cb4-bf1e-1317c1143d17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:52.207870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c8d99-5bf6-4cb4-bf1e-1317c1143d17 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:52.218268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1015/2016] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:52.218918 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03949c5f-e08e-437a-988f-823cb6c1a766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:52.220849 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03949c5f-e08e-437a-988f-823cb6c1a766 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:52.221144 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03949c5f-e08e-437a-988f-823cb6c1a766 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:52.221426 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03949c5f-e08e-437a-988f-823cb6c1a766 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:52.222195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03949c5f-e08e-437a-988f-823cb6c1a766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:52.223879 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03949c5f-e08e-437a-988f-823cb6c1a766 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:52.235607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1002/2017] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:52.239690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c8d99-5bf6-4cb4-bf1e-1317c1143d17 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:52.252746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c8d99-5bf6-4cb4-bf1e-1317c1143d17 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:52.261982 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a780f47-3f7b-4247-bb71-4d7d25b9e2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:52.263144 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a780f47-3f7b-4247-bb71-4d7d25b9e2b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:52.264710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a780f47-3f7b-4247-bb71-4d7d25b9e2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:52.265176 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a780f47-3f7b-4247-bb71-4d7d25b9e2b4 admin admin] PATH_INFO: `/v3/projects/c03684533b434d25b6b132c91e01505e/users/b7a346bbe3d14adeb18c55903aee642b/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:52.267523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a780f47-3f7b-4247-bb71-4d7d25b9e2b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03684533b434d25b6b132c91e01505e, user_id=b7a346bbe3d14adeb18c55903aee642b, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:52.270436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a780f47-3f7b-4247-bb71-4d7d25b9e2b4 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:52.311702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1016/2018] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:52 2022] PUT /identity/v3/projects/c03684533b434d25b6b132c91e01505e/users/b7a346bbe3d14adeb18c55903aee642b/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:52.322124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e6f3a-9c05-4a1a-8947-0e825c8c2007 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:52.322124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e6f3a-9c05-4a1a-8947-0e825c8c2007 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:52.322539 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e6f3a-9c05-4a1a-8947-0e825c8c2007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:52.371908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-365e6f3a-9c05-4a1a-8947-0e825c8c2007 None None] MFA Rules not processed for user `b7a346bbe3d14adeb18c55903aee642b`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:52.373721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-365e6f3a-9c05-4a1a-8947-0e825c8c2007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:52.417843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:52.499177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1003/2019] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:52 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:46:52.512371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a780f47-3f7b-4247-bb71-4d7d25b9e2b4 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:52.645874 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a780f47-3f7b-4247-bb71-4d7d25b9e2b4 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:52.657520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed32bb9-bfa1-4dca-a7d3-7d01f5fa1b97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:52.659948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed32bb9-bfa1-4dca-a7d3-7d01f5fa1b97 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:52.660797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed32bb9-bfa1-4dca-a7d3-7d01f5fa1b97 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:52.661497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed32bb9-bfa1-4dca-a7d3-7d01f5fa1b97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:52.686929 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed32bb9-bfa1-4dca-a7d3-7d01f5fa1b97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:52.689504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed32bb9-bfa1-4dca-a7d3-7d01f5fa1b97 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:52.710547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1017/2020] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:52 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:52.840134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e6f3a-9c05-4a1a-8947-0e825c8c2007 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:52.851077 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e6f3a-9c05-4a1a-8947-0e825c8c2007 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:52.855691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142eefe-2e3a-4fc8-9a85-26b3ebd4f72d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:52.856425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142eefe-2e3a-4fc8-9a85-26b3ebd4f72d service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:52.856755 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142eefe-2e3a-4fc8-9a85-26b3ebd4f72d service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:52.857041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142eefe-2e3a-4fc8-9a85-26b3ebd4f72d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:52.863736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed32bb9-bfa1-4dca-a7d3-7d01f5fa1b97 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:52.922378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142eefe-2e3a-4fc8-9a85-26b3ebd4f72d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:52.923834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142eefe-2e3a-4fc8-9a85-26b3ebd4f72d service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:52.941763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed32bb9-bfa1-4dca-a7d3-7d01f5fa1b97 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:52.944883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1004/2021] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:52.946531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:52.947483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:52.947832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:52.948202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] PATH_INFO: `/v3/users/9e60a97d2a1643d1b9c40deaee5b0c5c` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:52.950043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e60a97d2a1643d1b9c40deaee5b0c5c)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:52.952046 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:52.970550 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:53.015615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e60a97d2a1643d1b9c40deaee5b0c5c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.016284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e60a97d2a1643d1b9c40deaee5b0c5c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.024934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e60a97d2a1643d1b9c40deaee5b0c5c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.041823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e60a97d2a1643d1b9c40deaee5b0c5c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.049703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e60a97d2a1643d1b9c40deaee5b0c5c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.056243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1018/2022] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:52 2022] DELETE /identity/v3/users/9e60a97d2a1643d1b9c40deaee5b0c5c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:53.061806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38068d52-dcd0-461c-b241-4912f7021be0 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:53.062831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38068d52-dcd0-461c-b241-4912f7021be0 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:53.063215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38068d52-dcd0-461c-b241-4912f7021be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:53.085744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-38068d52-dcd0-461c-b241-4912f7021be0 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:53.087318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-38068d52-dcd0-461c-b241-4912f7021be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:53.188677 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1005/2023] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:53.200322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:53.202898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:53.325139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb76dd6-d780-4b73-b37a-d0dd3effb330 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:53.329271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3c845-2497-4bda-8b25-44e69d3aeef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:53.329945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3c845-2497-4bda-8b25-44e69d3aeef6 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:53.330433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3c845-2497-4bda-8b25-44e69d3aeef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:53.330716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3c845-2497-4bda-8b25-44e69d3aeef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:53.346009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3c845-2497-4bda-8b25-44e69d3aeef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:53.347728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3c845-2497-4bda-8b25-44e69d3aeef6 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:53.360456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1019/2024] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:53.642175 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38068d52-dcd0-461c-b241-4912f7021be0 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:53.644439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-38068d52-dcd0-461c-b241-4912f7021be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:53.666287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38068d52-dcd0-461c-b241-4912f7021be0 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:53.674766 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:53.675774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:53.675955 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:53.676266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] PATH_INFO: `/v3/projects/061203f8609740c2a80f6d6f10bb5421` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:53.677857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=061203f8609740c2a80f6d6f10bb5421)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:53.680748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:53.732644 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061203f8609740c2a80f6d6f10bb5421', 'request_id': 'req-855c4c31-32e5-497d-bd4d-e515591878d7'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.735561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:53.742704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061203f8609740c2a80f6d6f10bb5421', 'request_id': 'req-855c4c31-32e5-497d-bd4d-e515591878d7'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.749395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061203f8609740c2a80f6d6f10bb5421', 'request_id': 'req-855c4c31-32e5-497d-bd4d-e515591878d7'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.762093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] The token cache is being invalidate because project 061203f8609740c2a80f6d6f10bb5421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:53.762572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.763248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.765471 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1006/2025] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:53 2022] DELETE /identity/v3/projects/061203f8609740c2a80f6d6f10bb5421 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:53.769415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3c845-2497-4bda-8b25-44e69d3aeef6 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:53.781186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3c845-2497-4bda-8b25-44e69d3aeef6 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:53.847313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1cf49-b56f-43b2-820c-288b4d49f4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:53.848214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1cf49-b56f-43b2-820c-288b4d49f4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:53.848501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1cf49-b56f-43b2-820c-288b4d49f4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:53.848885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1cf49-b56f-43b2-820c-288b4d49f4b3 admin admin] PATH_INFO: `/v3/users/aa22756db4144f1cbdfe21ef8c7cca2d` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:53.850130 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1cf49-b56f-43b2-820c-288b4d49f4b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa22756db4144f1cbdfe21ef8c7cca2d)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:53.852098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1cf49-b56f-43b2-820c-288b4d49f4b3 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:53.904739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:53.912096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b4e1cf49-b56f-43b2-820c-288b4d49f4b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa22756db4144f1cbdfe21ef8c7cca2d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.913186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b4e1cf49-b56f-43b2-820c-288b4d49f4b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa22756db4144f1cbdfe21ef8c7cca2d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.923363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b4e1cf49-b56f-43b2-820c-288b4d49f4b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa22756db4144f1cbdfe21ef8c7cca2d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.939170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b4e1cf49-b56f-43b2-820c-288b4d49f4b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa22756db4144f1cbdfe21ef8c7cca2d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.948288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b4e1cf49-b56f-43b2-820c-288b4d49f4b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa22756db4144f1cbdfe21ef8c7cca2d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:53.955068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1020/2026] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:53 2022] DELETE /identity/v3/users/aa22756db4144f1cbdfe21ef8c7cca2d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:54.243891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:54.246014 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:54.279870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c4c31-32e5-497d-bd4d-e515591878d7 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:54.364549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9893c43-20f4-447a-aa86-22435e87919b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:54.365563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9893c43-20f4-447a-aa86-22435e87919b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:54.365939 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9893c43-20f4-447a-aa86-22435e87919b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:54.366311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9893c43-20f4-447a-aa86-22435e87919b admin admin] PATH_INFO: `/v3/projects/0653ece80d9f4c0e943937b1f4bc9784` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:54.367651 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9893c43-20f4-447a-aa86-22435e87919b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0653ece80d9f4c0e943937b1f4bc9784)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:54.369686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9893c43-20f4-447a-aa86-22435e87919b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:54.421407 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f9893c43-20f4-447a-aa86-22435e87919b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0653ece80d9f4c0e943937b1f4bc9784', 'request_id': 'req-f9893c43-20f4-447a-aa86-22435e87919b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:54.430733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f9893c43-20f4-447a-aa86-22435e87919b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0653ece80d9f4c0e943937b1f4bc9784', 'request_id': 'req-f9893c43-20f4-447a-aa86-22435e87919b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:54.441609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f9893c43-20f4-447a-aa86-22435e87919b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0653ece80d9f4c0e943937b1f4bc9784', 'request_id': 'req-f9893c43-20f4-447a-aa86-22435e87919b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:54.452864 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f9893c43-20f4-447a-aa86-22435e87919b admin admin] The token cache is being invalidate because project 0653ece80d9f4c0e943937b1f4bc9784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:46:54.453151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f9893c43-20f4-447a-aa86-22435e87919b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:54.453800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f9893c43-20f4-447a-aa86-22435e87919b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:54.455361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1007/2027] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:54 2022] DELETE /identity/v3/projects/0653ece80d9f4c0e943937b1f4bc9784 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:54.461191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde88c1-b389-431c-a76a-6b15ea5d7696 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:54.461487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde88c1-b389-431c-a76a-6b15ea5d7696 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:54.461770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde88c1-b389-431c-a76a-6b15ea5d7696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:54.482805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7fde88c1-b389-431c-a76a-6b15ea5d7696 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:54.484031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7fde88c1-b389-431c-a76a-6b15ea5d7696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:54.554379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1021/2028] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:54.555983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07c3fd-298f-4258-bac7-c35e2a36f7a1 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:54.556197 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07c3fd-298f-4258-bac7-c35e2a36f7a1 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:54.556510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07c3fd-298f-4258-bac7-c35e2a36f7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:54.579600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ab07c3fd-298f-4258-bac7-c35e2a36f7a1 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:54.580841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ab07c3fd-298f-4258-bac7-c35e2a36f7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:54.589513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1008/2029] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:54.593795 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde88c1-b389-431c-a76a-6b15ea5d7696 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:54.605552 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde88c1-b389-431c-a76a-6b15ea5d7696 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:54.612831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcff135-8c15-4ca9-b4fa-38c0aeb5d79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:54.613700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcff135-8c15-4ca9-b4fa-38c0aeb5d79a admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:54.614172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcff135-8c15-4ca9-b4fa-38c0aeb5d79a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:54.615622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcff135-8c15-4ca9-b4fa-38c0aeb5d79a admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:54.615919 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcff135-8c15-4ca9-b4fa-38c0aeb5d79a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:46:54.616351 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcff135-8c15-4ca9-b4fa-38c0aeb5d79a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:54.619101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcff135-8c15-4ca9-b4fa-38c0aeb5d79a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:54.643119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1022/2030] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:46:54.647755 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07c3fd-298f-4258-bac7-c35e2a36f7a1 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:54.657833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07c3fd-298f-4258-bac7-c35e2a36f7a1 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:54.663682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158eb3a7-c80f-4035-b9a6-8504987b80cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:54.664622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158eb3a7-c80f-4035-b9a6-8504987b80cb admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:54.664908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158eb3a7-c80f-4035-b9a6-8504987b80cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:54.665283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158eb3a7-c80f-4035-b9a6-8504987b80cb admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:54.665909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158eb3a7-c80f-4035-b9a6-8504987b80cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:54.667854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158eb3a7-c80f-4035-b9a6-8504987b80cb admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:54.700312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1009/2031] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:54 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:54.704785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcff135-8c15-4ca9-b4fa-38c0aeb5d79a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:54.715354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcff135-8c15-4ca9-b4fa-38c0aeb5d79a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:54.778596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd30ba-dd79-4acf-918b-eb6b982d7f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:54.779821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd30ba-dd79-4acf-918b-eb6b982d7f9a admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:54.780110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd30ba-dd79-4acf-918b-eb6b982d7f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:54.780388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd30ba-dd79-4acf-918b-eb6b982d7f9a admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:54.781144 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd30ba-dd79-4acf-918b-eb6b982d7f9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:54.783091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd30ba-dd79-4acf-918b-eb6b982d7f9a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:54.807343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1023/2032] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:54 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:46:54.811692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158eb3a7-c80f-4035-b9a6-8504987b80cb admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:54.822635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158eb3a7-c80f-4035-b9a6-8504987b80cb admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:54.827774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c21caa-42b4-4807-82fe-b1041954153b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:54.828559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c21caa-42b4-4807-82fe-b1041954153b admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:54.828955 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c21caa-42b4-4807-82fe-b1041954153b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:54.829285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c21caa-42b4-4807-82fe-b1041954153b admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:54.830038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c21caa-42b4-4807-82fe-b1041954153b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:54.831689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c21caa-42b4-4807-82fe-b1041954153b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:54.843231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1010/2033] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:46:54.847350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd30ba-dd79-4acf-918b-eb6b982d7f9a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:54.858565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd30ba-dd79-4acf-918b-eb6b982d7f9a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:54.863474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b8b5b-68c5-4257-8fa5-183a07138ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:54.864238 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b8b5b-68c5-4257-8fa5-183a07138ddf admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:54.864527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b8b5b-68c5-4257-8fa5-183a07138ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:54.864837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b8b5b-68c5-4257-8fa5-183a07138ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:54.865581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b8b5b-68c5-4257-8fa5-183a07138ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:54.867181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b8b5b-68c5-4257-8fa5-183a07138ddf admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:54.880619 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1024/2034] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:54.883327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c21caa-42b4-4807-82fe-b1041954153b admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:54.897691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c21caa-42b4-4807-82fe-b1041954153b admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:54.902537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512712f-1ebb-411d-aa07-6bd30b43fc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:54.903451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512712f-1ebb-411d-aa07-6bd30b43fc8d admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:54.903740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512712f-1ebb-411d-aa07-6bd30b43fc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:54.904561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512712f-1ebb-411d-aa07-6bd30b43fc8d admin admin] PATH_INFO: `/v3/projects/fce4bf2b5aa14cd7a09bf5d6e45462cf/users/a4d0598f67344b66a3f285da5f96962e/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:54.921324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512712f-1ebb-411d-aa07-6bd30b43fc8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fce4bf2b5aa14cd7a09bf5d6e45462cf, user_id=a4d0598f67344b66a3f285da5f96962e, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:54.924101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512712f-1ebb-411d-aa07-6bd30b43fc8d admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:54.939539 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1011/2035] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:54 2022] PUT /identity/v3/projects/fce4bf2b5aa14cd7a09bf5d6e45462cf/users/a4d0598f67344b66a3f285da5f96962e/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:54.950937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c99028-aa1e-45a4-9438-11b6c55e9beb None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:54.951255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c99028-aa1e-45a4-9438-11b6c55e9beb None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:54.951541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c99028-aa1e-45a4-9438-11b6c55e9beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:54.981076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-e9c99028-aa1e-45a4-9438-11b6c55e9beb None None] MFA Rules not processed for user `a4d0598f67344b66a3f285da5f96962e`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:54.982915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e9c99028-aa1e-45a4-9438-11b6c55e9beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:54.993222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:55.065828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1025/2036] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:55.070391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512712f-1ebb-411d-aa07-6bd30b43fc8d admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:55.080096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:55.140952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512712f-1ebb-411d-aa07-6bd30b43fc8d admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:55.150674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d47513-bbf2-4b48-98f1-a7cf07eba88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:55.151356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d47513-bbf2-4b48-98f1-a7cf07eba88c admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:55.151733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d47513-bbf2-4b48-98f1-a7cf07eba88c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:55.151914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d47513-bbf2-4b48-98f1-a7cf07eba88c admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:55.152651 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d47513-bbf2-4b48-98f1-a7cf07eba88c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:55.155293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d47513-bbf2-4b48-98f1-a7cf07eba88c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:55.184223 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1012/2037] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:46:55 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:55.188659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c99028-aa1e-45a4-9438-11b6c55e9beb None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:55.203202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c99028-aa1e-45a4-9438-11b6c55e9beb None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:55.266965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13793591-5a68-42f5-b41a-28cb5de873d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:55.267648 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13793591-5a68-42f5-b41a-28cb5de873d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:55.267936 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13793591-5a68-42f5-b41a-28cb5de873d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:55.268219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13793591-5a68-42f5-b41a-28cb5de873d1 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:55.268959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13793591-5a68-42f5-b41a-28cb5de873d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:55.270948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13793591-5a68-42f5-b41a-28cb5de873d1 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:55.294022 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1026/2038] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:46:55 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:46:55.299839 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d47513-bbf2-4b48-98f1-a7cf07eba88c admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:55.311051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d47513-bbf2-4b48-98f1-a7cf07eba88c admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:55.316404 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2853e30-893d-463c-a535-e93f572be7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:55.317060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2853e30-893d-463c-a535-e93f572be7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:55.317367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2853e30-893d-463c-a535-e93f572be7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:55.317775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2853e30-893d-463c-a535-e93f572be7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:55.318522 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2853e30-893d-463c-a535-e93f572be7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:55.320166 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2853e30-893d-463c-a535-e93f572be7d4 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:55.332038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1013/2039] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:55.336203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13793591-5a68-42f5-b41a-28cb5de873d1 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:55.347124 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13793591-5a68-42f5-b41a-28cb5de873d1 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:55.352689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e31a31-cf3a-4863-8b06-5e9017bfacb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:55.354811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e31a31-cf3a-4863-8b06-5e9017bfacb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:55.354811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e31a31-cf3a-4863-8b06-5e9017bfacb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:55.354811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e31a31-cf3a-4863-8b06-5e9017bfacb7 admin admin] PATH_INFO: `/v3/projects/8e0ea58f619c4dbb842c5de09e8992a5/users/98ca65d4243c4ef4839ec57dd07817e8/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:55.370881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e31a31-cf3a-4863-8b06-5e9017bfacb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0ea58f619c4dbb842c5de09e8992a5, user_id=98ca65d4243c4ef4839ec57dd07817e8, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:55.374774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e31a31-cf3a-4863-8b06-5e9017bfacb7 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:55.397070 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1027/2040] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:55 2022] PUT /identity/v3/projects/8e0ea58f619c4dbb842c5de09e8992a5/users/98ca65d4243c4ef4839ec57dd07817e8/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:55.400928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2853e30-893d-463c-a535-e93f572be7d4 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:55.413122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2853e30-893d-463c-a535-e93f572be7d4 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:55.488214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2766be7-2d90-4e46-aafb-2a66648b45ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:55.488927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2766be7-2d90-4e46-aafb-2a66648b45ae admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:55.489214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2766be7-2d90-4e46-aafb-2a66648b45ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:55.489492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2766be7-2d90-4e46-aafb-2a66648b45ae admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:55.490220 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2766be7-2d90-4e46-aafb-2a66648b45ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:55.491812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2766be7-2d90-4e46-aafb-2a66648b45ae admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:55.503027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1014/2041] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:46:55.507376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e31a31-cf3a-4863-8b06-5e9017bfacb7 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:55.518513 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e31a31-cf3a-4863-8b06-5e9017bfacb7 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:55.523209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afce19-45a4-491a-8fcb-906aa2f19678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:55.524089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afce19-45a4-491a-8fcb-906aa2f19678 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:55.524376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afce19-45a4-491a-8fcb-906aa2f19678 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:55.524685 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afce19-45a4-491a-8fcb-906aa2f19678 admin admin] PATH_INFO: `/v3/projects/8e0ea58f619c4dbb842c5de09e8992a5/users/98ca65d4243c4ef4839ec57dd07817e8/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:55.526667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afce19-45a4-491a-8fcb-906aa2f19678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0ea58f619c4dbb842c5de09e8992a5, user_id=98ca65d4243c4ef4839ec57dd07817e8, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:55.529392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afce19-45a4-491a-8fcb-906aa2f19678 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:55.539628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1028/2042] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:46:55 2022] PUT /identity/v3/projects/8e0ea58f619c4dbb842c5de09e8992a5/users/98ca65d4243c4ef4839ec57dd07817e8/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:46:55.549484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4ae93-0ccc-4903-80d7-cb7ab539752f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:55.549783 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4ae93-0ccc-4903-80d7-cb7ab539752f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:55.550070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4ae93-0ccc-4903-80d7-cb7ab539752f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:55.578526 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-afd4ae93-0ccc-4903-80d7-cb7ab539752f None None] MFA Rules not processed for user `98ca65d4243c4ef4839ec57dd07817e8`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:55.579750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-afd4ae93-0ccc-4903-80d7-cb7ab539752f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:55.649904 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1015/2043] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:55 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:55.661120 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afce19-45a4-491a-8fcb-906aa2f19678 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:55.662895 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f0afce19-45a4-491a-8fcb-906aa2f19678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:55.719680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afce19-45a4-491a-8fcb-906aa2f19678 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:55.723341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30584da4-3ba1-433a-a1a6-e1a549dde793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:55.723976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30584da4-3ba1-433a-a1a6-e1a549dde793 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:55.724261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30584da4-3ba1-433a-a1a6-e1a549dde793 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:55.724534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30584da4-3ba1-433a-a1a6-e1a549dde793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:55.734025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30584da4-3ba1-433a-a1a6-e1a549dde793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:55.735458 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30584da4-3ba1-433a-a1a6-e1a549dde793 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:55.745215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1029/2044] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:55.823052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4ae93-0ccc-4903-80d7-cb7ab539752f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:55.827857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:55.834938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4ae93-0ccc-4903-80d7-cb7ab539752f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:55.839106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69681c3d-c4f3-44c8-b01f-e1a857df957b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:55.839765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69681c3d-c4f3-44c8-b01f-e1a857df957b service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:55.840084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69681c3d-c4f3-44c8-b01f-e1a857df957b service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:55.840364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69681c3d-c4f3-44c8-b01f-e1a857df957b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:55.909941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69681c3d-c4f3-44c8-b01f-e1a857df957b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:55.911385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69681c3d-c4f3-44c8-b01f-e1a857df957b service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:55.921854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1016/2045] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:58.758350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30584da4-3ba1-433a-a1a6-e1a549dde793 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:58.767257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:46:58.823117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30584da4-3ba1-433a-a1a6-e1a549dde793 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:58.828328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:58.829225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:58.829579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:58.829784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] PATH_INFO: `/v3/users/a4d0598f67344b66a3f285da5f96962e` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:58.831114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4d0598f67344b66a3f285da5f96962e)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:58.834446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:58.886781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4d0598f67344b66a3f285da5f96962e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:58.887811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4d0598f67344b66a3f285da5f96962e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:58.898333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4d0598f67344b66a3f285da5f96962e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:58.912087 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4d0598f67344b66a3f285da5f96962e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:58.920981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4d0598f67344b66a3f285da5f96962e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:46:58.929299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1030/2046] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:46:58 2022] DELETE /identity/v3/users/a4d0598f67344b66a3f285da5f96962e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:46:58.933567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7c83a-3c29-4b0f-b01b-3b5a7f495182 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:58.934387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7c83a-3c29-4b0f-b01b-3b5a7f495182 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:58.934672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7c83a-3c29-4b0f-b01b-3b5a7f495182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:58.955551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-50f7c83a-3c29-4b0f-b01b-3b5a7f495182 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:46:58.956648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-50f7c83a-3c29-4b0f-b01b-3b5a7f495182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:59.161883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1017/2047] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:46:59.181155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:59.184051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:46:59.329467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f897e8e-8d06-4d70-a726-e964dcd58bff admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:46:59.344056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ba88d-5ba6-4053-9455-2ede727ff419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:46:59.344947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ba88d-5ba6-4053-9455-2ede727ff419 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:46:59.345235 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ba88d-5ba6-4053-9455-2ede727ff419 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:46:59.345514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ba88d-5ba6-4053-9455-2ede727ff419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:46:59.356378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ba88d-5ba6-4053-9455-2ede727ff419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:46:59.358468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ba88d-5ba6-4053-9455-2ede727ff419 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:46:59.373850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1031/2048] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:46:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:46:59.976954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7c83a-3c29-4b0f-b01b-3b5a7f495182 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:46:59.978998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-50f7c83a-3c29-4b0f-b01b-3b5a7f495182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:00.014710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7c83a-3c29-4b0f-b01b-3b5a7f495182 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:00.023414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:00.027309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:00.027473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:00.028083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] PATH_INFO: `/v3/projects/fce4bf2b5aa14cd7a09bf5d6e45462cf` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:00.029752 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] RBAC: Authorizing `identity:delete_project(project_id=fce4bf2b5aa14cd7a09bf5d6e45462cf)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:00.032167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:00.096251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fce4bf2b5aa14cd7a09bf5d6e45462cf', 'request_id': 'req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.109426 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fce4bf2b5aa14cd7a09bf5d6e45462cf', 'request_id': 'req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.128834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fce4bf2b5aa14cd7a09bf5d6e45462cf', 'request_id': 'req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.149060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] The token cache is being invalidate because project fce4bf2b5aa14cd7a09bf5d6e45462cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:47:00.149692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.150459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.153429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1018/2049] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:46:59 2022] DELETE /identity/v3/projects/fce4bf2b5aa14cd7a09bf5d6e45462cf => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:00.157916 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ba88d-5ba6-4053-9455-2ede727ff419 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:00.170059 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ba88d-5ba6-4053-9455-2ede727ff419 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:00.206216 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:00.239797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:00.241497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:00.251950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dd83c-3f7b-4118-94c0-dc2238587c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:00.253153 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dd83c-3f7b-4118-94c0-dc2238587c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:00.253598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dd83c-3f7b-4118-94c0-dc2238587c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:00.253881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dd83c-3f7b-4118-94c0-dc2238587c47 admin admin] PATH_INFO: `/v3/users/98ca65d4243c4ef4839ec57dd07817e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:00.255102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dd83c-3f7b-4118-94c0-dc2238587c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ca65d4243c4ef4839ec57dd07817e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:00.257608 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dd83c-3f7b-4118-94c0-dc2238587c47 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:00.273694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:00.333832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19c682-5fe7-4b0e-8160-6b82a0d4acba admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:00.335080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-642dd83c-3f7b-4118-94c0-dc2238587c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ca65d4243c4ef4839ec57dd07817e8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.336503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-642dd83c-3f7b-4118-94c0-dc2238587c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ca65d4243c4ef4839ec57dd07817e8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.339461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:00.340368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:00.340608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:00.340899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] PATH_INFO: `/v3/users/10953286aa6e4c88b59572f8e6eadbd1` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:00.342162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10953286aa6e4c88b59572f8e6eadbd1)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:00.344099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:00.345595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-642dd83c-3f7b-4118-94c0-dc2238587c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ca65d4243c4ef4839ec57dd07817e8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.359446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-642dd83c-3f7b-4118-94c0-dc2238587c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ca65d4243c4ef4839ec57dd07817e8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.366695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-642dd83c-3f7b-4118-94c0-dc2238587c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ca65d4243c4ef4839ec57dd07817e8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.374822 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1032/2050] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:47:00 2022] DELETE /identity/v3/users/98ca65d4243c4ef4839ec57dd07817e8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:00.395772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10953286aa6e4c88b59572f8e6eadbd1'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.405200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10953286aa6e4c88b59572f8e6eadbd1'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.423509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10953286aa6e4c88b59572f8e6eadbd1'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.433211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10953286aa6e4c88b59572f8e6eadbd1'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.434142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10953286aa6e4c88b59572f8e6eadbd1'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.443642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1019/2051] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:47:00 2022] DELETE /identity/v3/users/10953286aa6e4c88b59572f8e6eadbd1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:00.448684 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a911d-97d8-427e-a7b7-ef10f12c1eec None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:00.449028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a911d-97d8-427e-a7b7-ef10f12c1eec None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:00.449436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a911d-97d8-427e-a7b7-ef10f12c1eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:00.472856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-502a911d-97d8-427e-a7b7-ef10f12c1eec None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:00.474223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-502a911d-97d8-427e-a7b7-ef10f12c1eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:00.567702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1033/2052] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:00.574745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:00.576327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:00.683143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11a898-bba0-4c36-b49a-b4365046e5b8 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:00.689653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a14c5e-0bc6-494b-9da9-221e0a4d12e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:00.690722 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a14c5e-0bc6-494b-9da9-221e0a4d12e2 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:00.691218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a14c5e-0bc6-494b-9da9-221e0a4d12e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:00.691782 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a14c5e-0bc6-494b-9da9-221e0a4d12e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:00.724786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a14c5e-0bc6-494b-9da9-221e0a4d12e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:00.726476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a14c5e-0bc6-494b-9da9-221e0a4d12e2 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:00.745095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a911d-97d8-427e-a7b7-ef10f12c1eec None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:00.746684 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-502a911d-97d8-427e-a7b7-ef10f12c1eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:00.750451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1020/2053] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:47:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:47:00.760881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a911d-97d8-427e-a7b7-ef10f12c1eec None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:00.769960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b76ca-fe3a-4f1c-a089-f56471794058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:00.770895 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b76ca-fe3a-4f1c-a089-f56471794058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:00.771177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b76ca-fe3a-4f1c-a089-f56471794058 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:00.771604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b76ca-fe3a-4f1c-a089-f56471794058 admin admin] PATH_INFO: `/v3/projects/8e0ea58f619c4dbb842c5de09e8992a5` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:00.772859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b76ca-fe3a-4f1c-a089-f56471794058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e0ea58f619c4dbb842c5de09e8992a5)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:00.775980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b76ca-fe3a-4f1c-a089-f56471794058 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:00.826935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:00.831746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-090b76ca-fe3a-4f1c-a089-f56471794058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e0ea58f619c4dbb842c5de09e8992a5', 'request_id': 'req-090b76ca-fe3a-4f1c-a089-f56471794058'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.842892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-090b76ca-fe3a-4f1c-a089-f56471794058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e0ea58f619c4dbb842c5de09e8992a5', 'request_id': 'req-090b76ca-fe3a-4f1c-a089-f56471794058'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.851998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-090b76ca-fe3a-4f1c-a089-f56471794058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e0ea58f619c4dbb842c5de09e8992a5', 'request_id': 'req-090b76ca-fe3a-4f1c-a089-f56471794058'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.869711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-090b76ca-fe3a-4f1c-a089-f56471794058 admin admin] The token cache is being invalidate because project 8e0ea58f619c4dbb842c5de09e8992a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:47:00.870043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-090b76ca-fe3a-4f1c-a089-f56471794058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.871702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-090b76ca-fe3a-4f1c-a089-f56471794058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:00.876061 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1034/2054] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:47:00 2022] DELETE /identity/v3/projects/8e0ea58f619c4dbb842c5de09e8992a5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:00.885502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33947d8-9489-4be7-8cdb-a318f500513e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:00.885794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33947d8-9489-4be7-8cdb-a318f500513e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:00.886227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33947d8-9489-4be7-8cdb-a318f500513e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:00.944937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-b33947d8-9489-4be7-8cdb-a318f500513e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:00.949027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b33947d8-9489-4be7-8cdb-a318f500513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:01.007553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:01.031012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1021/2055] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:01.038657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dc9aa-4345-4674-9c19-b6a671550951 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.038948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dc9aa-4345-4674-9c19-b6a671550951 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.039228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dc9aa-4345-4674-9c19-b6a671550951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.059801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-663dc9aa-4345-4674-9c19-b6a671550951 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:01.061224 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-663dc9aa-4345-4674-9c19-b6a671550951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:01.069302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1035/2056] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:01.073534 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33947d8-9489-4be7-8cdb-a318f500513e None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:01.084300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33947d8-9489-4be7-8cdb-a318f500513e None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:01.089298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127e277-9630-4447-9fc7-f4cff9317836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:01.089973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127e277-9630-4447-9fc7-f4cff9317836 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.090260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127e277-9630-4447-9fc7-f4cff9317836 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.090565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127e277-9630-4447-9fc7-f4cff9317836 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.091869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127e277-9630-4447-9fc7-f4cff9317836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:47:01.092185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127e277-9630-4447-9fc7-f4cff9317836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:01.093846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127e277-9630-4447-9fc7-f4cff9317836 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:01.109460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1022/2057] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:47:01.117168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dc9aa-4345-4674-9c19-b6a671550951 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:01.128778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dc9aa-4345-4674-9c19-b6a671550951 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:01.136101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59e032-687c-4070-ad25-ba4b5c9123b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:01.136970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59e032-687c-4070-ad25-ba4b5c9123b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.137429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59e032-687c-4070-ad25-ba4b5c9123b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.137830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59e032-687c-4070-ad25-ba4b5c9123b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.139327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59e032-687c-4070-ad25-ba4b5c9123b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:01.141601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59e032-687c-4070-ad25-ba4b5c9123b4 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:01.193339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1036/2058] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:47:01 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:47:01.198209 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127e277-9630-4447-9fc7-f4cff9317836 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:01.237733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127e277-9630-4447-9fc7-f4cff9317836 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:01.357043 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc9a64-cbf5-4bce-8fd7-916fb3aa74bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:01.357926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc9a64-cbf5-4bce-8fd7-916fb3aa74bf admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.358327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc9a64-cbf5-4bce-8fd7-916fb3aa74bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.358797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc9a64-cbf5-4bce-8fd7-916fb3aa74bf admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.359775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc9a64-cbf5-4bce-8fd7-916fb3aa74bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:01.362570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc9a64-cbf5-4bce-8fd7-916fb3aa74bf admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:01.399023 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1023/2059] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:47:01 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:47:01.403354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59e032-687c-4070-ad25-ba4b5c9123b4 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:01.406050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0c59e032-687c-4070-ad25-ba4b5c9123b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:01.414074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc9a64-cbf5-4bce-8fd7-916fb3aa74bf admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:01.426250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc9a64-cbf5-4bce-8fd7-916fb3aa74bf admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:01.432418 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90e886-87ff-497f-9c71-66302f732387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:01.433999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90e886-87ff-497f-9c71-66302f732387 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.433999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90e886-87ff-497f-9c71-66302f732387 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.434204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90e886-87ff-497f-9c71-66302f732387 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.434865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90e886-87ff-497f-9c71-66302f732387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:01.437300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90e886-87ff-497f-9c71-66302f732387 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:01.443470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59e032-687c-4070-ad25-ba4b5c9123b4 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:01.453932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1024/2060] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:47:01.454910 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90e886-87ff-497f-9c71-66302f732387 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:01.455318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:01.456563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.456991 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.457515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] PATH_INFO: `/v3/projects/1c1987ef62cd4f9184b719cd827df24b` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.458971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c1987ef62cd4f9184b719cd827df24b)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:01.461014 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:01.469821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90e886-87ff-497f-9c71-66302f732387 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:01.474551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db09e06-4ea5-455f-9df8-3ec0c67b5b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:01.475274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db09e06-4ea5-455f-9df8-3ec0c67b5b8b admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.475563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db09e06-4ea5-455f-9df8-3ec0c67b5b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.475838 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db09e06-4ea5-455f-9df8-3ec0c67b5b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.476604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db09e06-4ea5-455f-9df8-3ec0c67b5b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:01.478179 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db09e06-4ea5-455f-9df8-3ec0c67b5b8b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:01.490951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1025/2061] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:47:01.491790 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db09e06-4ea5-455f-9df8-3ec0c67b5b8b admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:01.494532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-9db09e06-4ea5-455f-9df8-3ec0c67b5b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:01.508203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db09e06-4ea5-455f-9df8-3ec0c67b5b8b admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:01.519573 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c1987ef62cd4f9184b719cd827df24b', 'request_id': 'req-4cc215f8-e344-4861-9b78-26febdb5b507'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.528266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c1987ef62cd4f9184b719cd827df24b', 'request_id': 'req-4cc215f8-e344-4861-9b78-26febdb5b507'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.534853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c1987ef62cd4f9184b719cd827df24b', 'request_id': 'req-4cc215f8-e344-4861-9b78-26febdb5b507'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.547020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] The token cache is being invalidate because project 1c1987ef62cd4f9184b719cd827df24b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:47:01.547441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.548602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.550157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1037/2062] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:47:01 2022] DELETE /identity/v3/projects/1c1987ef62cd4f9184b719cd827df24b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:01.553035 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:01.553035 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:01.569063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32bb58-c5b9-4509-a602-ad686841b102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:01.569949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32bb58-c5b9-4509-a602-ad686841b102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.570238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32bb58-c5b9-4509-a602-ad686841b102 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.570521 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32bb58-c5b9-4509-a602-ad686841b102 admin admin] PATH_INFO: `/v3/users/f3c9b5d0309a4ecd9c724795b26c5174` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.571739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32bb58-c5b9-4509-a602-ad686841b102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3c9b5d0309a4ecd9c724795b26c5174)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:01.573707 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32bb58-c5b9-4509-a602-ad686841b102 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:01.618641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc215f8-e344-4861-9b78-26febdb5b507 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:01.619551 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:01.622952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-de32bb58-c5b9-4509-a602-ad686841b102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3c9b5d0309a4ecd9c724795b26c5174'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.624081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5503aab-9394-459d-97b6-d1b07621e8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:01.625114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5503aab-9394-459d-97b6-d1b07621e8fb admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.625455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5503aab-9394-459d-97b6-d1b07621e8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.625796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5503aab-9394-459d-97b6-d1b07621e8fb admin admin] PATH_INFO: `/v3/projects/d6a2c92dabea4b089aeaa684e1d97ed7/users/b891856026ac42fa8669e6ff451d184b/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.634795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-de32bb58-c5b9-4509-a602-ad686841b102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3c9b5d0309a4ecd9c724795b26c5174'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.646343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5503aab-9394-459d-97b6-d1b07621e8fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a2c92dabea4b089aeaa684e1d97ed7, user_id=b891856026ac42fa8669e6ff451d184b, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:01.649586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5503aab-9394-459d-97b6-d1b07621e8fb admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:01.663800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-de32bb58-c5b9-4509-a602-ad686841b102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3c9b5d0309a4ecd9c724795b26c5174'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.668665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1038/2063] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:47:01 2022] PUT /identity/v3/projects/d6a2c92dabea4b089aeaa684e1d97ed7/users/b891856026ac42fa8669e6ff451d184b/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:01.671203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5503aab-9394-459d-97b6-d1b07621e8fb admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:01.672535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-de32bb58-c5b9-4509-a602-ad686841b102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3c9b5d0309a4ecd9c724795b26c5174'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.673428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-de32bb58-c5b9-4509-a602-ad686841b102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3c9b5d0309a4ecd9c724795b26c5174'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.674071 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b5503aab-9394-459d-97b6-d1b07621e8fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:01.680535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1026/2064] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:47:01 2022] DELETE /identity/v3/users/f3c9b5d0309a4ecd9c724795b26c5174 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:01.681809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69353748-1474-4526-b649-3f47be307373 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.682096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69353748-1474-4526-b649-3f47be307373 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.682360 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69353748-1474-4526-b649-3f47be307373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.688188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:01.711008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-69353748-1474-4526-b649-3f47be307373 None None] MFA Rules not processed for user `b891856026ac42fa8669e6ff451d184b`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:01.712256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-69353748-1474-4526-b649-3f47be307373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:01.748339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5503aab-9394-459d-97b6-d1b07621e8fb admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:01.788610 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1027/2065] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:01.793713 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce39524-48b1-4b08-b2a7-4f699cad2d10 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.794001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce39524-48b1-4b08-b2a7-4f699cad2d10 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.794334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce39524-48b1-4b08-b2a7-4f699cad2d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.814711 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-0ce39524-48b1-4b08-b2a7-4f699cad2d10 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:01.816323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0ce39524-48b1-4b08-b2a7-4f699cad2d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:01.842543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:01.843551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.843844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.844126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] PATH_INFO: `/v3/users/a1fe2b0b143c45278ad5891300f83782` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.845601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1fe2b0b143c45278ad5891300f83782)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:01.847530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:01.886257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1028/2066] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:01.887327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce39524-48b1-4b08-b2a7-4f699cad2d10 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:01.889266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0ce39524-48b1-4b08-b2a7-4f699cad2d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:01.903463 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce39524-48b1-4b08-b2a7-4f699cad2d10 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:01.908796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29103fc0-c4e0-43f2-991d-7790ae5a097b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:01.909478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29103fc0-c4e0-43f2-991d-7790ae5a097b admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:01.911048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29103fc0-c4e0-43f2-991d-7790ae5a097b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:01.911048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29103fc0-c4e0-43f2-991d-7790ae5a097b admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:01.911048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29103fc0-c4e0-43f2-991d-7790ae5a097b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:01.913773 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29103fc0-c4e0-43f2-991d-7790ae5a097b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:01.931867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1fe2b0b143c45278ad5891300f83782'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.933008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1fe2b0b143c45278ad5891300f83782'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.945468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1fe2b0b143c45278ad5891300f83782'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.958681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1029/2067] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:47:01 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:47:01.958681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29103fc0-c4e0-43f2-991d-7790ae5a097b admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:01.962766 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-29103fc0-c4e0-43f2-991d-7790ae5a097b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:01.971752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1fe2b0b143c45278ad5891300f83782'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.980908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1fe2b0b143c45278ad5891300f83782'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:01.990989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1039/2068] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:47:01 2022] DELETE /identity/v3/users/a1fe2b0b143c45278ad5891300f83782 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:01.992072 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:01.994236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:02.011528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:02.053516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29103fc0-c4e0-43f2-991d-7790ae5a097b admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:02.056831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:02.102278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52771cc3-5267-447a-8e50-88bcb200da1c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:02.112463 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec298c8a-3d6f-47d5-b7bd-d0083592d934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:02.113429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec298c8a-3d6f-47d5-b7bd-d0083592d934 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:02.114507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec298c8a-3d6f-47d5-b7bd-d0083592d934 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:02.115442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec298c8a-3d6f-47d5-b7bd-d0083592d934 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:02.116363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec298c8a-3d6f-47d5-b7bd-d0083592d934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:02.118528 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec298c8a-3d6f-47d5-b7bd-d0083592d934 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:02.139080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99060ea-805c-45f6-8f17-fc9964c6908b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:02.140954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99060ea-805c-45f6-8f17-fc9964c6908b service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:02.142725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99060ea-805c-45f6-8f17-fc9964c6908b service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:02.142725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99060ea-805c-45f6-8f17-fc9964c6908b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:02.143837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a99060ea-805c-45f6-8f17-fc9964c6908b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:02.152608 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1040/2069] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:47:01 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:47:02.157503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec298c8a-3d6f-47d5-b7bd-d0083592d934 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:02.160214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99060ea-805c-45f6-8f17-fc9964c6908b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:02.161945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99060ea-805c-45f6-8f17-fc9964c6908b service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:02.179659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1030/2070] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:47:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:47:02.197555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec298c8a-3d6f-47d5-b7bd-d0083592d934 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:02.211628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cecbd-1bb4-4af2-8533-5e63d9b84042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:02.212180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cecbd-1bb4-4af2-8533-5e63d9b84042 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:02.212472 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cecbd-1bb4-4af2-8533-5e63d9b84042 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:02.212790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cecbd-1bb4-4af2-8533-5e63d9b84042 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:02.213539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592cecbd-1bb4-4af2-8533-5e63d9b84042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:02.215184 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592cecbd-1bb4-4af2-8533-5e63d9b84042 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:02.232970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1041/2071] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:47:02.234180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99060ea-805c-45f6-8f17-fc9964c6908b service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:02.257103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99060ea-805c-45f6-8f17-fc9964c6908b service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:02.267409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b973dad-8d38-43a6-b8c6-d9e43945d847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:02.269272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b973dad-8d38-43a6-b8c6-d9e43945d847 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:02.269645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b973dad-8d38-43a6-b8c6-d9e43945d847 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:02.270093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b973dad-8d38-43a6-b8c6-d9e43945d847 admin admin] PATH_INFO: `/v3/projects/d9f95b39da8a4461b302bbc6221d23ef/users/95a91ebc6538464893b20b55295c7240/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:02.309329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b973dad-8d38-43a6-b8c6-d9e43945d847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f95b39da8a4461b302bbc6221d23ef, user_id=95a91ebc6538464893b20b55295c7240, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:02.313950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b973dad-8d38-43a6-b8c6-d9e43945d847 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:02.338021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1031/2072] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:47:02 2022] PUT /identity/v3/projects/d9f95b39da8a4461b302bbc6221d23ef/users/95a91ebc6538464893b20b55295c7240/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:02.342497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cecbd-1bb4-4af2-8533-5e63d9b84042 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:02.354602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cecbd-1bb4-4af2-8533-5e63d9b84042 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:02.447073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941982e-0198-4a3a-90ee-976b6acfb48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:02.447840 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941982e-0198-4a3a-90ee-976b6acfb48a admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:02.448130 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941982e-0198-4a3a-90ee-976b6acfb48a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:02.448447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941982e-0198-4a3a-90ee-976b6acfb48a admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:02.449266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941982e-0198-4a3a-90ee-976b6acfb48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:02.451861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941982e-0198-4a3a-90ee-976b6acfb48a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:02.459686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b973dad-8d38-43a6-b8c6-d9e43945d847 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:02.461650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0b973dad-8d38-43a6-b8c6-d9e43945d847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:02.464828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1042/2073] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:47:02.469830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941982e-0198-4a3a-90ee-976b6acfb48a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:02.480685 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941982e-0198-4a3a-90ee-976b6acfb48a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:02.484353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b973dad-8d38-43a6-b8c6-d9e43945d847 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:02.486188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6ba46-e852-426c-9227-96b2854b4461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:02.487521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6ba46-e852-426c-9227-96b2854b4461 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:02.488043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6ba46-e852-426c-9227-96b2854b4461 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:02.488221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6ba46-e852-426c-9227-96b2854b4461 admin admin] PATH_INFO: `/v3/projects/d9f95b39da8a4461b302bbc6221d23ef/users/95a91ebc6538464893b20b55295c7240/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:02.491040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6ba46-e852-426c-9227-96b2854b4461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f95b39da8a4461b302bbc6221d23ef, user_id=95a91ebc6538464893b20b55295c7240, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:02.491265 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37588e-1b7f-4cc5-8c51-4e550213c921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:02.492621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37588e-1b7f-4cc5-8c51-4e550213c921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:02.492621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37588e-1b7f-4cc5-8c51-4e550213c921 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:02.494313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6ba46-e852-426c-9227-96b2854b4461 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:02.501482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37588e-1b7f-4cc5-8c51-4e550213c921 admin admin] PATH_INFO: `/v3/projects/bf5cb4a891a74bf486c3b0e5211f8fca` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:02.502984 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b37588e-1b7f-4cc5-8c51-4e550213c921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf5cb4a891a74bf486c3b0e5211f8fca)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:02.504900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b37588e-1b7f-4cc5-8c51-4e550213c921 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:02.509174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1043/2074] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:47:02 2022] PUT /identity/v3/projects/d9f95b39da8a4461b302bbc6221d23ef/users/95a91ebc6538464893b20b55295c7240/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:02.520520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8a14d-5e45-491f-8c19-e989533a0b34 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:02.522400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8a14d-5e45-491f-8c19-e989533a0b34 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:02.522941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8a14d-5e45-491f-8c19-e989533a0b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:02.558899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-28b8a14d-5e45-491f-8c19-e989533a0b34 None None] MFA Rules not processed for user `95a91ebc6538464893b20b55295c7240`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:02.560910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-28b8a14d-5e45-491f-8c19-e989533a0b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:02.578203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3b37588e-1b7f-4cc5-8c51-4e550213c921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf5cb4a891a74bf486c3b0e5211f8fca', 'request_id': 'req-3b37588e-1b7f-4cc5-8c51-4e550213c921'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:02.593503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3b37588e-1b7f-4cc5-8c51-4e550213c921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf5cb4a891a74bf486c3b0e5211f8fca', 'request_id': 'req-3b37588e-1b7f-4cc5-8c51-4e550213c921'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:02.602282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3b37588e-1b7f-4cc5-8c51-4e550213c921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf5cb4a891a74bf486c3b0e5211f8fca', 'request_id': 'req-3b37588e-1b7f-4cc5-8c51-4e550213c921'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:02.619608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3b37588e-1b7f-4cc5-8c51-4e550213c921 admin admin] The token cache is being invalidate because project bf5cb4a891a74bf486c3b0e5211f8fca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:47:02.619927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3b37588e-1b7f-4cc5-8c51-4e550213c921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:02.621325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3b37588e-1b7f-4cc5-8c51-4e550213c921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:02.624462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1032/2075] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:47:02 2022] DELETE /identity/v3/projects/bf5cb4a891a74bf486c3b0e5211f8fca => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:02.630461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800eafa-0441-4891-b286-36bd865dfb83 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:02.630918 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800eafa-0441-4891-b286-36bd865dfb83 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:02.631331 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800eafa-0441-4891-b286-36bd865dfb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:02.658048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1044/2076] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:02.670391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5800eafa-0441-4891-b286-36bd865dfb83 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:02.674301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5800eafa-0441-4891-b286-36bd865dfb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:02.677873 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8a14d-5e45-491f-8c19-e989533a0b34 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:02.679428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-28b8a14d-5e45-491f-8c19-e989533a0b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:02.713660 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:02.756879 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:02.760314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8a14d-5e45-491f-8c19-e989533a0b34 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:02.780604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1033/2077] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:02.781656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800eafa-0441-4891-b286-36bd865dfb83 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:02.783294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5800eafa-0441-4891-b286-36bd865dfb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:02.798630 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800eafa-0441-4891-b286-36bd865dfb83 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:02.804275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab3625-742b-47f6-ab7e-01391c48bb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:02.805182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab3625-742b-47f6-ab7e-01391c48bb82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:02.805466 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab3625-742b-47f6-ab7e-01391c48bb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:02.805747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab3625-742b-47f6-ab7e-01391c48bb82 admin admin] PATH_INFO: `/v3/projects/bb93e9f857fd4692b24ccf2b600ce511` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:02.806974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab3625-742b-47f6-ab7e-01391c48bb82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb93e9f857fd4692b24ccf2b600ce511)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:02.809113 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab3625-742b-47f6-ab7e-01391c48bb82 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:02.825666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8049a4d-3cb1-499b-b1b2-6b64b8081459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:02.826319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8049a4d-3cb1-499b-b1b2-6b64b8081459 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:02.826715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8049a4d-3cb1-499b-b1b2-6b64b8081459 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:02.827311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8049a4d-3cb1-499b-b1b2-6b64b8081459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:02.828982 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e8049a4d-3cb1-499b-b1b2-6b64b8081459 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:02.865258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ecab3625-742b-47f6-ab7e-01391c48bb82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb93e9f857fd4692b24ccf2b600ce511', 'request_id': 'req-ecab3625-742b-47f6-ab7e-01391c48bb82'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:02.886886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ecab3625-742b-47f6-ab7e-01391c48bb82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb93e9f857fd4692b24ccf2b600ce511', 'request_id': 'req-ecab3625-742b-47f6-ab7e-01391c48bb82'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:02.897739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ecab3625-742b-47f6-ab7e-01391c48bb82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb93e9f857fd4692b24ccf2b600ce511', 'request_id': 'req-ecab3625-742b-47f6-ab7e-01391c48bb82'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:02.900449 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8049a4d-3cb1-499b-b1b2-6b64b8081459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:02.901949 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8049a4d-3cb1-499b-b1b2-6b64b8081459 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:02.910648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ecab3625-742b-47f6-ab7e-01391c48bb82 admin admin] The token cache is being invalidate because project bb93e9f857fd4692b24ccf2b600ce511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:47:02.910873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ecab3625-742b-47f6-ab7e-01391c48bb82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:02.911808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ecab3625-742b-47f6-ab7e-01391c48bb82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:02.916219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1034/2078] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:47:02 2022] DELETE /identity/v3/projects/bb93e9f857fd4692b24ccf2b600ce511 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:02.916219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788bf26-8fd6-42ca-90d1-a58fe4fd7203 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:02.916219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788bf26-8fd6-42ca-90d1-a58fe4fd7203 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:02.916219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788bf26-8fd6-42ca-90d1-a58fe4fd7203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:02.916621 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1045/2079] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:47:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:47:02.917750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8049a4d-3cb1-499b-b1b2-6b64b8081459 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:02.931606 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8049a4d-3cb1-499b-b1b2-6b64b8081459 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:02.940938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3788bf26-8fd6-42ca-90d1-a58fe4fd7203 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:02.942313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3788bf26-8fd6-42ca-90d1-a58fe4fd7203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:03.025513 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:03.027502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:03.027797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:03.028081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] PATH_INFO: `/v3/users/b7a346bbe3d14adeb18c55903aee642b` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:03.030607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a346bbe3d14adeb18c55903aee642b)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:03.033322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:03.065007 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1035/2080] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:03.070271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788bf26-8fd6-42ca-90d1-a58fe4fd7203 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:03.080612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788bf26-8fd6-42ca-90d1-a58fe4fd7203 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:03.085502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:03.088287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bac169-a979-471b-a90d-c1f6dafc8fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:03.089497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bac169-a979-471b-a90d-c1f6dafc8fde admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:03.090147 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bac169-a979-471b-a90d-c1f6dafc8fde admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:03.090572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bac169-a979-471b-a90d-c1f6dafc8fde admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:03.091891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bac169-a979-471b-a90d-c1f6dafc8fde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:47:03.092206 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bac169-a979-471b-a90d-c1f6dafc8fde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:03.094655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bac169-a979-471b-a90d-c1f6dafc8fde admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:03.101985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a346bbe3d14adeb18c55903aee642b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:03.102923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a346bbe3d14adeb18c55903aee642b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:03.116924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1036/2081] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:47:03.117279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a346bbe3d14adeb18c55903aee642b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:03.121510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bac169-a979-471b-a90d-c1f6dafc8fde admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:03.132747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bac169-a979-471b-a90d-c1f6dafc8fde admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:03.138876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736c93c-b7f6-4045-9ea1-00d1b271f6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:03.139348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a346bbe3d14adeb18c55903aee642b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:03.140046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736c93c-b7f6-4045-9ea1-00d1b271f6ae admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:03.140402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736c93c-b7f6-4045-9ea1-00d1b271f6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:03.140846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736c93c-b7f6-4045-9ea1-00d1b271f6ae admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:03.141591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736c93c-b7f6-4045-9ea1-00d1b271f6ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:03.143878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736c93c-b7f6-4045-9ea1-00d1b271f6ae admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:03.149981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a346bbe3d14adeb18c55903aee642b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:03.160739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1046/2082] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:47:02 2022] DELETE /identity/v3/users/b7a346bbe3d14adeb18c55903aee642b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:03.179256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1037/2083] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:47:03 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:47:03.183333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:03.184920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:03.300631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41620728-8bcb-40b3-8ce7-b085fa0df67d admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:03.425319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8862f-9f76-426d-be44-3b0c05be6301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:03.425889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8862f-9f76-426d-be44-3b0c05be6301 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:03.426260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8862f-9f76-426d-be44-3b0c05be6301 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:03.426621 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8862f-9f76-426d-be44-3b0c05be6301 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:03.427673 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8862f-9f76-426d-be44-3b0c05be6301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:03.429675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8862f-9f76-426d-be44-3b0c05be6301 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:03.456613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1047/2084] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:47:03 2022] POST /identity/v3/users => generated 288 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:47:03.458712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736c93c-b7f6-4045-9ea1-00d1b271f6ae admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:03.471440 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736c93c-b7f6-4045-9ea1-00d1b271f6ae admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:03.476988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ead3c-894c-4cd0-b0e7-988ff5ded4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:03.477798 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ead3c-894c-4cd0-b0e7-988ff5ded4b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:03.478185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ead3c-894c-4cd0-b0e7-988ff5ded4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:03.478563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ead3c-894c-4cd0-b0e7-988ff5ded4b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:03.479529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ead3c-894c-4cd0-b0e7-988ff5ded4b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:03.481272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ead3c-894c-4cd0-b0e7-988ff5ded4b2 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:03.498387 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8862f-9f76-426d-be44-3b0c05be6301 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:03.498747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1038/2085] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:47:03.511126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8862f-9f76-426d-be44-3b0c05be6301 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:03.516982 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d216d-668f-46e5-8337-3af2efc257fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:03.517586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d216d-668f-46e5-8337-3af2efc257fb admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:03.517870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d216d-668f-46e5-8337-3af2efc257fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:03.518146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d216d-668f-46e5-8337-3af2efc257fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:03.518887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d216d-668f-46e5-8337-3af2efc257fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:03.521278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d216d-668f-46e5-8337-3af2efc257fb admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:03.534214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1048/2086] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:47:03.538322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ead3c-894c-4cd0-b0e7-988ff5ded4b2 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:03.551123 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ead3c-894c-4cd0-b0e7-988ff5ded4b2 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:03.557624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db046ad4-23a4-4bcf-b546-0b1d738fe3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:03.558736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db046ad4-23a4-4bcf-b546-0b1d738fe3c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:03.559171 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db046ad4-23a4-4bcf-b546-0b1d738fe3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:03.559595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db046ad4-23a4-4bcf-b546-0b1d738fe3c0 admin admin] PATH_INFO: `/v3/projects/37f782edd19b4a16bc09ab67cf59e80d/users/42908415b3a7494f971bd9f333e6a90a/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:03.578499 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db046ad4-23a4-4bcf-b546-0b1d738fe3c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f782edd19b4a16bc09ab67cf59e80d, user_id=42908415b3a7494f971bd9f333e6a90a, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:03.581233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db046ad4-23a4-4bcf-b546-0b1d738fe3c0 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:03.611067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:03.618011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1039/2087] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:47:03 2022] PUT /identity/v3/projects/37f782edd19b4a16bc09ab67cf59e80d/users/42908415b3a7494f971bd9f333e6a90a/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:03.628146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4559dcd-c7cc-4374-b600-3b0adcfcba76 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:03.628824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4559dcd-c7cc-4374-b600-3b0adcfcba76 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:03.629307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4559dcd-c7cc-4374-b600-3b0adcfcba76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:03.667560 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a4559dcd-c7cc-4374-b600-3b0adcfcba76 None None] MFA Rules not processed for user `42908415b3a7494f971bd9f333e6a90a`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:03.669492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a4559dcd-c7cc-4374-b600-3b0adcfcba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:03.706651 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:03.750285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1049/2088] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:47:03.762484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db046ad4-23a4-4bcf-b546-0b1d738fe3c0 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:03.764931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-db046ad4-23a4-4bcf-b546-0b1d738fe3c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:03.802946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4559dcd-c7cc-4374-b600-3b0adcfcba76 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:03.804495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a4559dcd-c7cc-4374-b600-3b0adcfcba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:03.846361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db046ad4-23a4-4bcf-b546-0b1d738fe3c0 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:03.850612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8167a3-078a-4e86-b57e-8ded1cf789e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:03.851364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8167a3-078a-4e86-b57e-8ded1cf789e5 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:03.851729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8167a3-078a-4e86-b57e-8ded1cf789e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:03.852117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8167a3-078a-4e86-b57e-8ded1cf789e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:03.862836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8167a3-078a-4e86-b57e-8ded1cf789e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:03.864355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8167a3-078a-4e86-b57e-8ded1cf789e5 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:03.876856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1040/2089] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:47:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:47:03.915835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4559dcd-c7cc-4374-b600-3b0adcfcba76 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:03.921483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f31d0-657c-417f-86a9-ee48cf80c4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:03.923125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f31d0-657c-417f-86a9-ee48cf80c4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:03.923418 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f31d0-657c-417f-86a9-ee48cf80c4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:03.923697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f31d0-657c-417f-86a9-ee48cf80c4d8 admin admin] PATH_INFO: `/v3/projects/c03684533b434d25b6b132c91e01505e` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:03.924955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f31d0-657c-417f-86a9-ee48cf80c4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c03684533b434d25b6b132c91e01505e)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:03.927212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f31d0-657c-417f-86a9-ee48cf80c4d8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:03.982472 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a21f31d0-657c-417f-86a9-ee48cf80c4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c03684533b434d25b6b132c91e01505e', 'request_id': 'req-a21f31d0-657c-417f-86a9-ee48cf80c4d8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:03.990837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a21f31d0-657c-417f-86a9-ee48cf80c4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c03684533b434d25b6b132c91e01505e', 'request_id': 'req-a21f31d0-657c-417f-86a9-ee48cf80c4d8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:03.996899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a21f31d0-657c-417f-86a9-ee48cf80c4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c03684533b434d25b6b132c91e01505e', 'request_id': 'req-a21f31d0-657c-417f-86a9-ee48cf80c4d8'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:04.008413 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a21f31d0-657c-417f-86a9-ee48cf80c4d8 admin admin] The token cache is being invalidate because project c03684533b434d25b6b132c91e01505e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:47:04.008739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a21f31d0-657c-417f-86a9-ee48cf80c4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:04.009404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a21f31d0-657c-417f-86a9-ee48cf80c4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:04.011234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1050/2090] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:47:03 2022] DELETE /identity/v3/projects/c03684533b434d25b6b132c91e01505e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:47:04.016121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02b565-0bc7-42ef-b2d1-758980685e5e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:04.016429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02b565-0bc7-42ef-b2d1-758980685e5e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:04.016767 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02b565-0bc7-42ef-b2d1-758980685e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:04.037084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3f02b565-0bc7-42ef-b2d1-758980685e5e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:04.038326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3f02b565-0bc7-42ef-b2d1-758980685e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:04.112182 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8425bb-763f-4766-b4b2-600d2c85c822 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:04.112483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8425bb-763f-4766-b4b2-600d2c85c822 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:04.112799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8425bb-763f-4766-b4b2-600d2c85c822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:04.116183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1041/2091] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:04.133209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7e8425bb-763f-4766-b4b2-600d2c85c822 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:04.134750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7e8425bb-763f-4766-b4b2-600d2c85c822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:04.147954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02b565-0bc7-42ef-b2d1-758980685e5e None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:04.156754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1051/2092] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:04.165405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02b565-0bc7-42ef-b2d1-758980685e5e None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:04.171650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9fbeb-9b78-4378-ad3c-30ba7e545b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:04.172403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9fbeb-9b78-4378-ad3c-30ba7e545b89 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:04.172617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9fbeb-9b78-4378-ad3c-30ba7e545b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:04.172984 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9fbeb-9b78-4378-ad3c-30ba7e545b89 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:04.173835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9fbeb-9b78-4378-ad3c-30ba7e545b89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:47:04.174153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9fbeb-9b78-4378-ad3c-30ba7e545b89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:04.177118 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9fbeb-9b78-4378-ad3c-30ba7e545b89 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:04.192889 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1042/2093] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:47:04.197300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8425bb-763f-4766-b4b2-600d2c85c822 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:04.208538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8425bb-763f-4766-b4b2-600d2c85c822 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:04.214431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72089c7-0ee8-4e9a-9acc-027f80027c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:04.215538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72089c7-0ee8-4e9a-9acc-027f80027c80 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:04.215828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72089c7-0ee8-4e9a-9acc-027f80027c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:04.216174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72089c7-0ee8-4e9a-9acc-027f80027c80 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:04.217794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72089c7-0ee8-4e9a-9acc-027f80027c80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:04.219754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72089c7-0ee8-4e9a-9acc-027f80027c80 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:04.256620 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1052/2094] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:47:04 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:47:04.259401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9fbeb-9b78-4378-ad3c-30ba7e545b89 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:04.271162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9fbeb-9b78-4378-ad3c-30ba7e545b89 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:04.329467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04527c0f-7b9b-459e-af7e-1c8d329e6900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:04.330142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04527c0f-7b9b-459e-af7e-1c8d329e6900 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:04.330427 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04527c0f-7b9b-459e-af7e-1c8d329e6900 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:04.330706 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04527c0f-7b9b-459e-af7e-1c8d329e6900 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:04.331415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04527c0f-7b9b-459e-af7e-1c8d329e6900 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:04.333371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04527c0f-7b9b-459e-af7e-1c8d329e6900 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:04.354642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1043/2095] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:47:04 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:47:04.359345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72089c7-0ee8-4e9a-9acc-027f80027c80 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:04.370724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72089c7-0ee8-4e9a-9acc-027f80027c80 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:04.376088 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5a499-db7f-413c-bad9-e33d964f8378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:04.376826 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5a499-db7f-413c-bad9-e33d964f8378 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:04.377110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5a499-db7f-413c-bad9-e33d964f8378 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:04.377410 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5a499-db7f-413c-bad9-e33d964f8378 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:04.378147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5a499-db7f-413c-bad9-e33d964f8378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:04.379741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5a499-db7f-413c-bad9-e33d964f8378 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:04.390745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1053/2096] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:47:04.394764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04527c0f-7b9b-459e-af7e-1c8d329e6900 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:04.405175 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04527c0f-7b9b-459e-af7e-1c8d329e6900 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:04.409880 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce49fe-188a-4b13-948b-0741ba2b1441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:04.410537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce49fe-188a-4b13-948b-0741ba2b1441 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:04.410823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce49fe-188a-4b13-948b-0741ba2b1441 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:04.411100 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce49fe-188a-4b13-948b-0741ba2b1441 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:04.411827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce49fe-188a-4b13-948b-0741ba2b1441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:04.413538 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce49fe-188a-4b13-948b-0741ba2b1441 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:04.425383 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1044/2097] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:47:04.429534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5a499-db7f-413c-bad9-e33d964f8378 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:04.441555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5a499-db7f-413c-bad9-e33d964f8378 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:04.446212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04830aa-4e92-4156-8d97-2b9ba0c9556e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:04.447109 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04830aa-4e92-4156-8d97-2b9ba0c9556e admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:04.447391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04830aa-4e92-4156-8d97-2b9ba0c9556e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:04.447663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04830aa-4e92-4156-8d97-2b9ba0c9556e admin admin] PATH_INFO: `/v3/projects/4f935a736a61408b8eb2801ff91227d2/users/757acf3fbe3d49bd9bd98be72924a661/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:04.462072 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04830aa-4e92-4156-8d97-2b9ba0c9556e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f935a736a61408b8eb2801ff91227d2, user_id=757acf3fbe3d49bd9bd98be72924a661, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:04.464740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04830aa-4e92-4156-8d97-2b9ba0c9556e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:04.467452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:04.479039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1054/2098] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:47:04 2022] PUT /identity/v3/projects/4f935a736a61408b8eb2801ff91227d2/users/757acf3fbe3d49bd9bd98be72924a661/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:04.490561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b3896-83df-42a7-a3e6-f506b58beb0b None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:04.490851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b3896-83df-42a7-a3e6-f506b58beb0b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:04.491132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b3896-83df-42a7-a3e6-f506b58beb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:04.511175 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:04.520797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-2f7b3896-83df-42a7-a3e6-f506b58beb0b None None] MFA Rules not processed for user `757acf3fbe3d49bd9bd98be72924a661`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:04.521984 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2f7b3896-83df-42a7-a3e6-f506b58beb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:04.584504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1045/2099] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:04.595822 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04830aa-4e92-4156-8d97-2b9ba0c9556e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:04.597595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f04830aa-4e92-4156-8d97-2b9ba0c9556e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:04.658902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04830aa-4e92-4156-8d97-2b9ba0c9556e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:04.662773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85399ef6-67a5-4bdb-ad29-4666f68c5683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:04.663529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85399ef6-67a5-4bdb-ad29-4666f68c5683 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:04.663922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85399ef6-67a5-4bdb-ad29-4666f68c5683 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:04.664303 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85399ef6-67a5-4bdb-ad29-4666f68c5683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:04.673121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85399ef6-67a5-4bdb-ad29-4666f68c5683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:04.674557 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85399ef6-67a5-4bdb-ad29-4666f68c5683 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:04.683675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1055/2100] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:47:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:47:14.581648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=28 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:14.679698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:15.964136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b3896-83df-42a7-a3e6-f506b58beb0b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:15.966118 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2f7b3896-83df-42a7-a3e6-f506b58beb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:15.977687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b3896-83df-42a7-a3e6-f506b58beb0b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:15.981923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e3a51-11b5-4048-aa37-8326421b71f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:15.982566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e3a51-11b5-4048-aa37-8326421b71f0 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:15.982848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e3a51-11b5-4048-aa37-8326421b71f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:15.983124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e3a51-11b5-4048-aa37-8326421b71f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:15.984849 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e06e3a51-11b5-4048-aa37-8326421b71f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:16.048085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e3a51-11b5-4048-aa37-8326421b71f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:16.049536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e3a51-11b5-4048-aa37-8326421b71f0 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:16.058914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1046/2101] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:47:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:47:26.054684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=30 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:36.750839 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85399ef6-67a5-4bdb-ad29-4666f68c5683 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:36.818988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85399ef6-67a5-4bdb-ad29-4666f68c5683 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:36.823544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:36.824422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:36.824818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:36.825099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] PATH_INFO: `/v3/users/757acf3fbe3d49bd9bd98be72924a661` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:36.826325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] RBAC: Authorizing `identity:delete_user(user_id=757acf3fbe3d49bd9bd98be72924a661)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:36.828897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:36.873222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '757acf3fbe3d49bd9bd98be72924a661'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:36.874408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '757acf3fbe3d49bd9bd98be72924a661'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:36.881901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '757acf3fbe3d49bd9bd98be72924a661'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:36.894432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '757acf3fbe3d49bd9bd98be72924a661'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:36.901103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '757acf3fbe3d49bd9bd98be72924a661'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:36.906996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1056/2102] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:47:36 2022] DELETE /identity/v3/users/757acf3fbe3d49bd9bd98be72924a661 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:36.911580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7e42d-3494-4f89-8c31-8fffe70c2a52 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:36.912040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7e42d-3494-4f89-8c31-8fffe70c2a52 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:36.912132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7e42d-3494-4f89-8c31-8fffe70c2a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:36.930554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3ea7e42d-3494-4f89-8c31-8fffe70c2a52 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:36.931733 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3ea7e42d-3494-4f89-8c31-8fffe70c2a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:36.989510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1047/2103] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:36.998940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:37.000434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:37.060316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88df818-3bfe-4437-b9aa-011bdf25b33a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:37.063985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354eae05-9635-4e77-a0b7-d8871d3b3c23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:37.064670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354eae05-9635-4e77-a0b7-d8871d3b3c23 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:37.064955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354eae05-9635-4e77-a0b7-d8871d3b3c23 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:37.065233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354eae05-9635-4e77-a0b7-d8871d3b3c23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:37.074647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354eae05-9635-4e77-a0b7-d8871d3b3c23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:37.076075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354eae05-9635-4e77-a0b7-d8871d3b3c23 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:37.086759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1057/2104] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:47:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:47:37.430552 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7e42d-3494-4f89-8c31-8fffe70c2a52 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:37.432179 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3ea7e42d-3494-4f89-8c31-8fffe70c2a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:37.447396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7e42d-3494-4f89-8c31-8fffe70c2a52 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:37.474425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f6b3d-a661-42ac-b739-8ca5919446ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:37.475350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6b3d-a661-42ac-b739-8ca5919446ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:37.475709 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6b3d-a661-42ac-b739-8ca5919446ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:37.476044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6b3d-a661-42ac-b739-8ca5919446ed admin admin] PATH_INFO: `/v3/projects/4f935a736a61408b8eb2801ff91227d2` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:37.477276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f6b3d-a661-42ac-b739-8ca5919446ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f935a736a61408b8eb2801ff91227d2)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:37.479372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f6b3d-a661-42ac-b739-8ca5919446ed admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:37.528508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ac5f6b3d-a661-42ac-b739-8ca5919446ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f935a736a61408b8eb2801ff91227d2', 'request_id': 'req-ac5f6b3d-a661-42ac-b739-8ca5919446ed'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:37.536731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ac5f6b3d-a661-42ac-b739-8ca5919446ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f935a736a61408b8eb2801ff91227d2', 'request_id': 'req-ac5f6b3d-a661-42ac-b739-8ca5919446ed'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:37.542456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ac5f6b3d-a661-42ac-b739-8ca5919446ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f935a736a61408b8eb2801ff91227d2', 'request_id': 'req-ac5f6b3d-a661-42ac-b739-8ca5919446ed'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:37.553934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ac5f6b3d-a661-42ac-b739-8ca5919446ed admin admin] The token cache is being invalidate because project 4f935a736a61408b8eb2801ff91227d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:47:37.554218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ac5f6b3d-a661-42ac-b739-8ca5919446ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:37.554853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ac5f6b3d-a661-42ac-b739-8ca5919446ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:37.556386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1048/2105] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:47:37 2022] DELETE /identity/v3/projects/4f935a736a61408b8eb2801ff91227d2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:37.561474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3109a3-3c90-4acb-8f5f-721c86601dd2 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:37.561860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3109a3-3c90-4acb-8f5f-721c86601dd2 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:37.562210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3109a3-3c90-4acb-8f5f-721c86601dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:37.582098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-9a3109a3-3c90-4acb-8f5f-721c86601dd2 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:37.583417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9a3109a3-3c90-4acb-8f5f-721c86601dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:37.601296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:37.649177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1058/2106] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:37.657274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a94c-fbf5-439d-b1b0-397a06e8ec76 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:37.657563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a94c-fbf5-439d-b1b0-397a06e8ec76 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:37.657843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a94c-fbf5-439d-b1b0-397a06e8ec76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:37.677554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-f481a94c-fbf5-439d-b1b0-397a06e8ec76 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:37.678740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f481a94c-fbf5-439d-b1b0-397a06e8ec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:37.686655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1049/2107] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:37.690764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3109a3-3c90-4acb-8f5f-721c86601dd2 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:37.700548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3109a3-3c90-4acb-8f5f-721c86601dd2 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:37.705140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3069f-54d7-451e-bc4d-1d77b7ab989e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:37.705801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3069f-54d7-451e-bc4d-1d77b7ab989e admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:37.706090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3069f-54d7-451e-bc4d-1d77b7ab989e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:37.706372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3069f-54d7-451e-bc4d-1d77b7ab989e admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:37.707106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3069f-54d7-451e-bc4d-1d77b7ab989e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:47:37.707422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3069f-54d7-451e-bc4d-1d77b7ab989e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:37.709141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3069f-54d7-451e-bc4d-1d77b7ab989e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:37.721329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1059/2108] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:47:37.725890 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481a94c-fbf5-439d-b1b0-397a06e8ec76 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:37.735690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481a94c-fbf5-439d-b1b0-397a06e8ec76 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:37.740324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c4c59-d8ae-4af2-8fed-ea15f783cb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:37.741007 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c4c59-d8ae-4af2-8fed-ea15f783cb05 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:37.741292 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c4c59-d8ae-4af2-8fed-ea15f783cb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:37.741571 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c4c59-d8ae-4af2-8fed-ea15f783cb05 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:37.742272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c4c59-d8ae-4af2-8fed-ea15f783cb05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:37.744195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c4c59-d8ae-4af2-8fed-ea15f783cb05 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:37.771272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1050/2109] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:47:37 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:47:37.775349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3069f-54d7-451e-bc4d-1d77b7ab989e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:37.785935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3069f-54d7-451e-bc4d-1d77b7ab989e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:37.843352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcc791-fed6-4790-a33f-d4c7b18b8eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:37.844037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcc791-fed6-4790-a33f-d4c7b18b8eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:37.844329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcc791-fed6-4790-a33f-d4c7b18b8eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:37.844634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcc791-fed6-4790-a33f-d4c7b18b8eb3 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:37.845341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcc791-fed6-4790-a33f-d4c7b18b8eb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:37.847291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcc791-fed6-4790-a33f-d4c7b18b8eb3 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:37.873469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1060/2110] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:47:37 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:47:37.874943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c4c59-d8ae-4af2-8fed-ea15f783cb05 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:37.886343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c4c59-d8ae-4af2-8fed-ea15f783cb05 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:37.891491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccffc19-4cc4-49c7-a4f0-cb273e1c06c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:37.892156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccffc19-4cc4-49c7-a4f0-cb273e1c06c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:37.892440 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccffc19-4cc4-49c7-a4f0-cb273e1c06c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:37.892755 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccffc19-4cc4-49c7-a4f0-cb273e1c06c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:37.893486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccffc19-4cc4-49c7-a4f0-cb273e1c06c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:37.895082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccffc19-4cc4-49c7-a4f0-cb273e1c06c3 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:37.905064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1051/2111] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:47:37.909643 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcc791-fed6-4790-a33f-d4c7b18b8eb3 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:37.919615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcc791-fed6-4790-a33f-d4c7b18b8eb3 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:37.924189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d87516-0bf9-454f-94ce-334eeecd2aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:37.925163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d87516-0bf9-454f-94ce-334eeecd2aca admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:37.925456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d87516-0bf9-454f-94ce-334eeecd2aca admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:37.925735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d87516-0bf9-454f-94ce-334eeecd2aca admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:37.926492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d87516-0bf9-454f-94ce-334eeecd2aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:37.928073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d87516-0bf9-454f-94ce-334eeecd2aca admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:37.937950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1061/2112] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:47:37.942012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccffc19-4cc4-49c7-a4f0-cb273e1c06c3 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:37.952417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccffc19-4cc4-49c7-a4f0-cb273e1c06c3 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:37.957084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a69a54-f50c-4107-bbef-e473b954a3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:37.957955 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a69a54-f50c-4107-bbef-e473b954a3ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:37.958236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a69a54-f50c-4107-bbef-e473b954a3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:37.958512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a69a54-f50c-4107-bbef-e473b954a3ac admin admin] PATH_INFO: `/v3/projects/1a07844f474c4846aec8863a7d1d6e04/users/7719a37809e74cfbb6d00b2ac3714a97/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:37.973625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a69a54-f50c-4107-bbef-e473b954a3ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a07844f474c4846aec8863a7d1d6e04, user_id=7719a37809e74cfbb6d00b2ac3714a97, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:37.976258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a69a54-f50c-4107-bbef-e473b954a3ac admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:37.988942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1052/2113] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:47:37 2022] PUT /identity/v3/projects/1a07844f474c4846aec8863a7d1d6e04/users/7719a37809e74cfbb6d00b2ac3714a97/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:38.000089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fef37-f96c-45c7-9e3e-c17d3d49df1e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:38.000386 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fef37-f96c-45c7-9e3e-c17d3d49df1e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:38.000705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fef37-f96c-45c7-9e3e-c17d3d49df1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:38.028537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-b77fef37-f96c-45c7-9e3e-c17d3d49df1e None None] MFA Rules not processed for user `7719a37809e74cfbb6d00b2ac3714a97`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:38.029856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b77fef37-f96c-45c7-9e3e-c17d3d49df1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:38.094294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1062/2114] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:38.105132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a69a54-f50c-4107-bbef-e473b954a3ac admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:38.107514 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-82a69a54-f50c-4107-bbef-e473b954a3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:38.131409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:38.168747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a69a54-f50c-4107-bbef-e473b954a3ac admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:38.172465 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872ba63-adf9-4fb3-a553-ded2d99a4751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:38.173160 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872ba63-adf9-4fb3-a553-ded2d99a4751 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:38.173449 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872ba63-adf9-4fb3-a553-ded2d99a4751 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:38.173732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872ba63-adf9-4fb3-a553-ded2d99a4751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:38.182002 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872ba63-adf9-4fb3-a553-ded2d99a4751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:38.183444 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872ba63-adf9-4fb3-a553-ded2d99a4751 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:38.192697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1053/2115] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:47:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:47:48.090696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:48.188702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=17 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:57.006837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fef37-f96c-45c7-9e3e-c17d3d49df1e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:57.090204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fef37-f96c-45c7-9e3e-c17d3d49df1e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:57.097626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:57.098976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:57.099273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:57.099553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] PATH_INFO: `/v3/users/7719a37809e74cfbb6d00b2ac3714a97` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:57.100846 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7719a37809e74cfbb6d00b2ac3714a97)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:57.102901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:57.175631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7719a37809e74cfbb6d00b2ac3714a97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:57.176709 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7719a37809e74cfbb6d00b2ac3714a97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:57.189813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7719a37809e74cfbb6d00b2ac3714a97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:57.210547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7719a37809e74cfbb6d00b2ac3714a97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:57.219780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7719a37809e74cfbb6d00b2ac3714a97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:57.226735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1063/2116] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:47:57 2022] DELETE /identity/v3/users/7719a37809e74cfbb6d00b2ac3714a97 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:57.230926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972f77c-6ae2-47b8-b638-22af2801aca4 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:57.231411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972f77c-6ae2-47b8-b638-22af2801aca4 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:57.231512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972f77c-6ae2-47b8-b638-22af2801aca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:57.251217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-1972f77c-6ae2-47b8-b638-22af2801aca4 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:57.252417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1972f77c-6ae2-47b8-b638-22af2801aca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:57.318882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1054/2117] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:57.328897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:57.330448 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:57.396225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067e41d-9995-4253-bff3-6f8781527414 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:57.400803 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17e784-4df0-4860-b561-0529f2ba7d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:57.401530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17e784-4df0-4860-b561-0529f2ba7d51 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:57.401817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17e784-4df0-4860-b561-0529f2ba7d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:57.402149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17e784-4df0-4860-b561-0529f2ba7d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:57.413152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17e784-4df0-4860-b561-0529f2ba7d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:57.414683 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17e784-4df0-4860-b561-0529f2ba7d51 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:57.425871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1064/2118] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:47:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:47:57.875818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972f77c-6ae2-47b8-b638-22af2801aca4 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:57.877174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1972f77c-6ae2-47b8-b638-22af2801aca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:57.895322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972f77c-6ae2-47b8-b638-22af2801aca4 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:57.901883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f94b2-5f02-4f5a-85cb-a1a51c61048e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:57.925363 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f94b2-5f02-4f5a-85cb-a1a51c61048e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:57.925734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f94b2-5f02-4f5a-85cb-a1a51c61048e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:57.928324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f94b2-5f02-4f5a-85cb-a1a51c61048e admin admin] PATH_INFO: `/v3/projects/1a07844f474c4846aec8863a7d1d6e04` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:57.932661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f94b2-5f02-4f5a-85cb-a1a51c61048e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a07844f474c4846aec8863a7d1d6e04)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:57.934682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f94b2-5f02-4f5a-85cb-a1a51c61048e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:58.035240 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-930f94b2-5f02-4f5a-85cb-a1a51c61048e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a07844f474c4846aec8863a7d1d6e04', 'request_id': 'req-930f94b2-5f02-4f5a-85cb-a1a51c61048e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:58.045371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-930f94b2-5f02-4f5a-85cb-a1a51c61048e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a07844f474c4846aec8863a7d1d6e04', 'request_id': 'req-930f94b2-5f02-4f5a-85cb-a1a51c61048e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:58.052694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-930f94b2-5f02-4f5a-85cb-a1a51c61048e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a07844f474c4846aec8863a7d1d6e04', 'request_id': 'req-930f94b2-5f02-4f5a-85cb-a1a51c61048e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:58.064467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-930f94b2-5f02-4f5a-85cb-a1a51c61048e admin admin] The token cache is being invalidate because project 1a07844f474c4846aec8863a7d1d6e04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:47:58.064811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-930f94b2-5f02-4f5a-85cb-a1a51c61048e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:58.065519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-930f94b2-5f02-4f5a-85cb-a1a51c61048e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:47:58.067273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1055/2119] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:47:57 2022] DELETE /identity/v3/projects/1a07844f474c4846aec8863a7d1d6e04 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:47:58.072078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc06e5-8cd5-4d7f-9a83-b63a48b5225a None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:58.072610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc06e5-8cd5-4d7f-9a83-b63a48b5225a None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:58.072744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc06e5-8cd5-4d7f-9a83-b63a48b5225a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:58.092963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d2dc06e5-8cd5-4d7f-9a83-b63a48b5225a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:58.095637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d2dc06e5-8cd5-4d7f-9a83-b63a48b5225a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:58.132478 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:58.200622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1065/2120] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:58.208199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0d719-f6b9-4d60-ac41-48875825c11a None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:58.208490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0d719-f6b9-4d60-ac41-48875825c11a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:58.208820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0d719-f6b9-4d60-ac41-48875825c11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:58.236930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-0fd0d719-f6b9-4d60-ac41-48875825c11a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:58.238148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0fd0d719-f6b9-4d60-ac41-48875825c11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:58.250492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1056/2121] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:58.255089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc06e5-8cd5-4d7f-9a83-b63a48b5225a None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:58.286595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc06e5-8cd5-4d7f-9a83-b63a48b5225a None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:58.298923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1651e-a659-407c-adac-5e3bb4ce85e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:58.299626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1651e-a659-407c-adac-5e3bb4ce85e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:58.299917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1651e-a659-407c-adac-5e3bb4ce85e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:58.300264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1651e-a659-407c-adac-5e3bb4ce85e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:58.307730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1651e-a659-407c-adac-5e3bb4ce85e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:47:58.307888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1651e-a659-407c-adac-5e3bb4ce85e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:58.309990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1651e-a659-407c-adac-5e3bb4ce85e0 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:58.330962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1066/2122] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:47:58.334203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0d719-f6b9-4d60-ac41-48875825c11a None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:58.344557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0d719-f6b9-4d60-ac41-48875825c11a None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:58.349812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cff77f-ba21-471b-8cd3-c44b972e52ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:58.350762 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cff77f-ba21-471b-8cd3-c44b972e52ef admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:58.351226 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cff77f-ba21-471b-8cd3-c44b972e52ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:58.351688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cff77f-ba21-471b-8cd3-c44b972e52ef admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:58.352975 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cff77f-ba21-471b-8cd3-c44b972e52ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:58.355170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cff77f-ba21-471b-8cd3-c44b972e52ef admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:58.401235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1057/2123] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:47:58 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:47:58.407512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1651e-a659-407c-adac-5e3bb4ce85e0 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:58.424750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1651e-a659-407c-adac-5e3bb4ce85e0 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:58.496145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a740e1-80df-497c-a35e-4b6a9bc4cac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:58.496931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a740e1-80df-497c-a35e-4b6a9bc4cac3 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:58.497282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a740e1-80df-497c-a35e-4b6a9bc4cac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:58.497628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a740e1-80df-497c-a35e-4b6a9bc4cac3 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:58.498397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a740e1-80df-497c-a35e-4b6a9bc4cac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:58.501088 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a740e1-80df-497c-a35e-4b6a9bc4cac3 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:58.525113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1067/2124] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:47:58 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:47:58.529829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cff77f-ba21-471b-8cd3-c44b972e52ef admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:58.540848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cff77f-ba21-471b-8cd3-c44b972e52ef admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:58.546268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d3f85-e166-471a-89e6-565d3e13e497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:58.546939 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d3f85-e166-471a-89e6-565d3e13e497 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:58.547245 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d3f85-e166-471a-89e6-565d3e13e497 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:58.547497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d3f85-e166-471a-89e6-565d3e13e497 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:58.548405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d3f85-e166-471a-89e6-565d3e13e497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:58.550325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d3f85-e166-471a-89e6-565d3e13e497 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:58.560719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1058/2125] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:47:58.565392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a740e1-80df-497c-a35e-4b6a9bc4cac3 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:58.575841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a740e1-80df-497c-a35e-4b6a9bc4cac3 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:58.581840 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c142269-bdb7-46aa-8857-36231f75067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:58.582502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c142269-bdb7-46aa-8857-36231f75067c admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:58.582795 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c142269-bdb7-46aa-8857-36231f75067c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:58.583076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c142269-bdb7-46aa-8857-36231f75067c admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:58.583808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c142269-bdb7-46aa-8857-36231f75067c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:58.585519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c142269-bdb7-46aa-8857-36231f75067c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:58.601263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1068/2126] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:47:58.602251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d3f85-e166-471a-89e6-565d3e13e497 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:58.613049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d3f85-e166-471a-89e6-565d3e13e497 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:58.620914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bb209-a5df-44c0-b5b8-14ba585ca815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:58.621820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bb209-a5df-44c0-b5b8-14ba585ca815 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:58.622106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bb209-a5df-44c0-b5b8-14ba585ca815 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:58.622546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bb209-a5df-44c0-b5b8-14ba585ca815 admin admin] PATH_INFO: `/v3/projects/778a06bd709a4e3abe691e5624ec6d4a/users/0cf75c04abc5478f9557908bd94afd9d/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:58.642307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bb209-a5df-44c0-b5b8-14ba585ca815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778a06bd709a4e3abe691e5624ec6d4a, user_id=0cf75c04abc5478f9557908bd94afd9d, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:58.645679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bb209-a5df-44c0-b5b8-14ba585ca815 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:58.660522 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1059/2127] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:47:58 2022] PUT /identity/v3/projects/778a06bd709a4e3abe691e5624ec6d4a/users/0cf75c04abc5478f9557908bd94afd9d/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:47:58.670789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66c8d0-e58b-4b41-b004-1013c5379175 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:58.671116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66c8d0-e58b-4b41-b004-1013c5379175 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:58.671407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66c8d0-e58b-4b41-b004-1013c5379175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:58.701985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-2f66c8d0-e58b-4b41-b004-1013c5379175 None None] MFA Rules not processed for user `0cf75c04abc5478f9557908bd94afd9d`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:58.703917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2f66c8d0-e58b-4b41-b004-1013c5379175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:58.784127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1069/2128] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:47:58.795216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bb209-a5df-44c0-b5b8-14ba585ca815 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:47:58.797560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7b0bb209-a5df-44c0-b5b8-14ba585ca815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:58.826180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:58.866741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bb209-a5df-44c0-b5b8-14ba585ca815 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:47:58.871219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda69e80-34e3-4b14-9082-02df6c4983c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:47:58.871915 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda69e80-34e3-4b14-9082-02df6c4983c2 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:58.872272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda69e80-34e3-4b14-9082-02df6c4983c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:58.872557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda69e80-34e3-4b14-9082-02df6c4983c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:58.883546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda69e80-34e3-4b14-9082-02df6c4983c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:47:58.885018 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda69e80-34e3-4b14-9082-02df6c4983c2 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:47:58.894577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1060/2129] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:47:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:47:59.560762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c65ef-52e4-4685-bedd-16c357477d31 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:47:59.561289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c65ef-52e4-4685-bedd-16c357477d31 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:47:59.561490 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c65ef-52e4-4685-bedd-16c357477d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:47:59.582577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-142c65ef-52e4-4685-bedd-16c357477d31 None None] MFA Rules not processed for user `bf0c0131372345e6b23260b466ec8ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:47:59.583934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-142c65ef-52e4-4685-bedd-16c357477d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:47:59.597783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:47:59.643712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1070/2130] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:47:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:06.892698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda69e80-34e3-4b14-9082-02df6c4983c2 service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:06.895175 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-bda69e80-34e3-4b14-9082-02df6c4983c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:07.042945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda69e80-34e3-4b14-9082-02df6c4983c2 service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:07.054067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:07.055067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:07.056876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:07.057126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] PATH_INFO: `/v3/users/42908415b3a7494f971bd9f333e6a90a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:07.058483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42908415b3a7494f971bd9f333e6a90a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:07.074650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:07.245202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42908415b3a7494f971bd9f333e6a90a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:07.277483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42908415b3a7494f971bd9f333e6a90a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:07.317332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42908415b3a7494f971bd9f333e6a90a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:07.331404 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42908415b3a7494f971bd9f333e6a90a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:07.332339 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42908415b3a7494f971bd9f333e6a90a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:07.340553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1061/2131] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:06 2022] DELETE /identity/v3/users/42908415b3a7494f971bd9f333e6a90a => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:07.350504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e961b-7dd6-424e-b8b0-42354a1c0fff None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:07.351010 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e961b-7dd6-424e-b8b0-42354a1c0fff None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:07.351130 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e961b-7dd6-424e-b8b0-42354a1c0fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:07.384215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-0b6e961b-7dd6-424e-b8b0-42354a1c0fff None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:07.385775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0b6e961b-7dd6-424e-b8b0-42354a1c0fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:07.502447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1071/2132] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:07.534596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:07.536669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:07.637415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:07.654720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e0f8b-469f-453c-b325-b110006cfb88 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:07.659549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbe92b-6c7e-49b9-9f4e-22ad0f66924b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:07.660202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbe92b-6c7e-49b9-9f4e-22ad0f66924b service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:07.660484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbe92b-6c7e-49b9-9f4e-22ad0f66924b service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:07.660800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbe92b-6c7e-49b9-9f4e-22ad0f66924b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:07.673674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbe92b-6c7e-49b9-9f4e-22ad0f66924b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:07.677319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbe92b-6c7e-49b9-9f4e-22ad0f66924b service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:07.691670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1062/2133] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:08.129224 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e961b-7dd6-424e-b8b0-42354a1c0fff None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:08.131300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0b6e961b-7dd6-424e-b8b0-42354a1c0fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:08.145271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e961b-7dd6-424e-b8b0-42354a1c0fff None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:08.150129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e8159-63e6-47e2-9e37-d4f9bee83f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:08.151001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e8159-63e6-47e2-9e37-d4f9bee83f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:08.151289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e8159-63e6-47e2-9e37-d4f9bee83f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:08.151573 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e8159-63e6-47e2-9e37-d4f9bee83f20 admin admin] PATH_INFO: `/v3/projects/37f782edd19b4a16bc09ab67cf59e80d` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:08.152832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e8159-63e6-47e2-9e37-d4f9bee83f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37f782edd19b4a16bc09ab67cf59e80d)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:08.154766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e8159-63e6-47e2-9e37-d4f9bee83f20 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:08.211823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-152e8159-63e6-47e2-9e37-d4f9bee83f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37f782edd19b4a16bc09ab67cf59e80d', 'request_id': 'req-152e8159-63e6-47e2-9e37-d4f9bee83f20'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:08.224388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-152e8159-63e6-47e2-9e37-d4f9bee83f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37f782edd19b4a16bc09ab67cf59e80d', 'request_id': 'req-152e8159-63e6-47e2-9e37-d4f9bee83f20'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:08.230910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-152e8159-63e6-47e2-9e37-d4f9bee83f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37f782edd19b4a16bc09ab67cf59e80d', 'request_id': 'req-152e8159-63e6-47e2-9e37-d4f9bee83f20'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:08.242899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-152e8159-63e6-47e2-9e37-d4f9bee83f20 admin admin] The token cache is being invalidate because project 37f782edd19b4a16bc09ab67cf59e80d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:08.243161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-152e8159-63e6-47e2-9e37-d4f9bee83f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:08.243891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-152e8159-63e6-47e2-9e37-d4f9bee83f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:08.245863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1072/2134] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:08 2022] DELETE /identity/v3/projects/37f782edd19b4a16bc09ab67cf59e80d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:08.251281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e9d53-ec0f-450c-8701-9a35ea7fd770 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:08.251730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e9d53-ec0f-450c-8701-9a35ea7fd770 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:08.251869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e9d53-ec0f-450c-8701-9a35ea7fd770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:08.272622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5e2e9d53-ec0f-450c-8701-9a35ea7fd770 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:08.274102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5e2e9d53-ec0f-450c-8701-9a35ea7fd770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:08.369604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1063/2135] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:08.378161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc412e-263e-4cd2-8677-7f4ac0328a6e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:08.378161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc412e-263e-4cd2-8677-7f4ac0328a6e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:08.378682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc412e-263e-4cd2-8677-7f4ac0328a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:08.434138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-2cfc412e-263e-4cd2-8677-7f4ac0328a6e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:08.440033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2cfc412e-263e-4cd2-8677-7f4ac0328a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:08.458527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1073/2136] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:08.462389 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e9d53-ec0f-450c-8701-9a35ea7fd770 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:08.473957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e9d53-ec0f-450c-8701-9a35ea7fd770 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:08.497447 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bb2b0-96ec-4e2f-9862-eae7a2523897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:08.498016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bb2b0-96ec-4e2f-9862-eae7a2523897 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:08.498297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bb2b0-96ec-4e2f-9862-eae7a2523897 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:08.499334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bb2b0-96ec-4e2f-9862-eae7a2523897 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:08.500090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bb2b0-96ec-4e2f-9862-eae7a2523897 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:48:08.500403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bb2b0-96ec-4e2f-9862-eae7a2523897 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:08.504391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bb2b0-96ec-4e2f-9862-eae7a2523897 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:08.527072 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1064/2137] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:48:08.527530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc412e-263e-4cd2-8677-7f4ac0328a6e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:08.544031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc412e-263e-4cd2-8677-7f4ac0328a6e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:08.558712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045e3af-888e-4a1e-8a74-c21b97f02af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:08.559691 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045e3af-888e-4a1e-8a74-c21b97f02af0 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:08.560184 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045e3af-888e-4a1e-8a74-c21b97f02af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:08.560688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045e3af-888e-4a1e-8a74-c21b97f02af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:08.562322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045e3af-888e-4a1e-8a74-c21b97f02af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:08.566846 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045e3af-888e-4a1e-8a74-c21b97f02af0 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:08.614597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1074/2138] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:08 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:08.618431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bb2b0-96ec-4e2f-9862-eae7a2523897 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:08.632167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bb2b0-96ec-4e2f-9862-eae7a2523897 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:08.762721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3e660-83f5-4589-bb44-0fbd042b91eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:08.763089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3e660-83f5-4589-bb44-0fbd042b91eb admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:08.763489 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3e660-83f5-4589-bb44-0fbd042b91eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:08.763846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3e660-83f5-4589-bb44-0fbd042b91eb admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:08.764908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3e660-83f5-4589-bb44-0fbd042b91eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:08.768137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3e660-83f5-4589-bb44-0fbd042b91eb admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:08.800483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1065/2139] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:08 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:08.804477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045e3af-888e-4a1e-8a74-c21b97f02af0 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:08.816361 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045e3af-888e-4a1e-8a74-c21b97f02af0 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:08.831877 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6a8ae-5300-4377-a4e5-99868a693fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:08.832654 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6a8ae-5300-4377-a4e5-99868a693fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:08.833839 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6a8ae-5300-4377-a4e5-99868a693fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:08.834346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6a8ae-5300-4377-a4e5-99868a693fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:08.835542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6a8ae-5300-4377-a4e5-99868a693fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:08.837765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6a8ae-5300-4377-a4e5-99868a693fb9 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:08.876716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1075/2140] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:48:08.880743 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3e660-83f5-4589-bb44-0fbd042b91eb admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:08.894307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3e660-83f5-4589-bb44-0fbd042b91eb admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:08.900353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb15839-9a12-45b8-a156-abfe1dbd400d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:08.901222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb15839-9a12-45b8-a156-abfe1dbd400d admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:08.901770 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb15839-9a12-45b8-a156-abfe1dbd400d admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:08.902069 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb15839-9a12-45b8-a156-abfe1dbd400d admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:08.903468 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb15839-9a12-45b8-a156-abfe1dbd400d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:08.908466 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb15839-9a12-45b8-a156-abfe1dbd400d admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:08.921724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1066/2141] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:08.926054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6a8ae-5300-4377-a4e5-99868a693fb9 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:08.937843 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6a8ae-5300-4377-a4e5-99868a693fb9 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:08.942609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc1623-013e-48b4-818f-f440601de98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:08.943529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc1623-013e-48b4-818f-f440601de98e admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:08.943830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc1623-013e-48b4-818f-f440601de98e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:08.944100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc1623-013e-48b4-818f-f440601de98e admin admin] PATH_INFO: `/v3/projects/062fa2d348eb41b59501286b11233bca/users/209152d018ef450e97ca5e6955db0009/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:08.946534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:08.963395 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc1623-013e-48b4-818f-f440601de98e admin admin] RBAC: Authorizing `identity:create_grant(project_id=062fa2d348eb41b59501286b11233bca, user_id=209152d018ef450e97ca5e6955db0009, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:08.966132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc1623-013e-48b4-818f-f440601de98e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:08.978729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1076/2142] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:08 2022] PUT /identity/v3/projects/062fa2d348eb41b59501286b11233bca/users/209152d018ef450e97ca5e6955db0009/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:08.990170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc3417-0ef4-4d4a-be55-77ee14aa6f67 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:08.990467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc3417-0ef4-4d4a-be55-77ee14aa6f67 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:08.990781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc3417-0ef4-4d4a-be55-77ee14aa6f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.017979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-2afc3417-0ef4-4d4a-be55-77ee14aa6f67 None None] MFA Rules not processed for user `209152d018ef450e97ca5e6955db0009`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:09.019189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2afc3417-0ef4-4d4a-be55-77ee14aa6f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:09.059494 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:09.084000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1067/2143] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:09.088585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc1623-013e-48b4-818f-f440601de98e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:09.149187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc1623-013e-48b4-818f-f440601de98e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:09.153752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96b25f-db90-474c-a20d-51aa49ba50df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:09.154414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96b25f-db90-474c-a20d-51aa49ba50df admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.154695 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96b25f-db90-474c-a20d-51aa49ba50df admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.155493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96b25f-db90-474c-a20d-51aa49ba50df admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.156218 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96b25f-db90-474c-a20d-51aa49ba50df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:09.158165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96b25f-db90-474c-a20d-51aa49ba50df admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:09.184441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1077/2144] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:09 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:48:09.188619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc3417-0ef4-4d4a-be55-77ee14aa6f67 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:09.199186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc3417-0ef4-4d4a-be55-77ee14aa6f67 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:09.257084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03fe78-d6be-4cf8-b5b9-68f00ee38c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:09.257765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03fe78-d6be-4cf8-b5b9-68f00ee38c54 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.258305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03fe78-d6be-4cf8-b5b9-68f00ee38c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.258578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03fe78-d6be-4cf8-b5b9-68f00ee38c54 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.259324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03fe78-d6be-4cf8-b5b9-68f00ee38c54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:09.261313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03fe78-d6be-4cf8-b5b9-68f00ee38c54 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:09.284139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1068/2145] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:09 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:48:09.287984 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96b25f-db90-474c-a20d-51aa49ba50df admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:09.299353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96b25f-db90-474c-a20d-51aa49ba50df admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:09.304331 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d51d1-b16e-4527-9de4-4f1441b0d91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:09.305053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d51d1-b16e-4527-9de4-4f1441b0d91a admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.305330 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d51d1-b16e-4527-9de4-4f1441b0d91a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.305635 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d51d1-b16e-4527-9de4-4f1441b0d91a admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.306372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d51d1-b16e-4527-9de4-4f1441b0d91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:09.307958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d51d1-b16e-4527-9de4-4f1441b0d91a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:09.318832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1078/2146] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:09.322805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03fe78-d6be-4cf8-b5b9-68f00ee38c54 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:09.335536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03fe78-d6be-4cf8-b5b9-68f00ee38c54 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:09.341057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee29474-515e-4978-8beb-75a6f139a4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:09.341863 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee29474-515e-4978-8beb-75a6f139a4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.342225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee29474-515e-4978-8beb-75a6f139a4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.343233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee29474-515e-4978-8beb-75a6f139a4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.343978 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee29474-515e-4978-8beb-75a6f139a4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:09.345634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee29474-515e-4978-8beb-75a6f139a4c8 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:09.355676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1069/2147] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:09.359989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d51d1-b16e-4527-9de4-4f1441b0d91a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:09.370016 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d51d1-b16e-4527-9de4-4f1441b0d91a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:09.374730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82677081-60c9-4843-ad31-88f03b050a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:09.375613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82677081-60c9-4843-ad31-88f03b050a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.375898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82677081-60c9-4843-ad31-88f03b050a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.376179 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82677081-60c9-4843-ad31-88f03b050a4f admin admin] PATH_INFO: `/v3/projects/a36b528e07d148b3bc9c24a96b34c738/users/9d717fc739eb4a1fb84e53a06b5e9ff6/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.391592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82677081-60c9-4843-ad31-88f03b050a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a36b528e07d148b3bc9c24a96b34c738, user_id=9d717fc739eb4a1fb84e53a06b5e9ff6, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:09.394307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82677081-60c9-4843-ad31-88f03b050a4f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:09.408425 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1079/2148] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:09 2022] PUT /identity/v3/projects/a36b528e07d148b3bc9c24a96b34c738/users/9d717fc739eb4a1fb84e53a06b5e9ff6/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:09.419676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab0637-7e16-4c54-b0e3-3b3a7fbfd428 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.419964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab0637-7e16-4c54-b0e3-3b3a7fbfd428 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.420243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab0637-7e16-4c54-b0e3-3b3a7fbfd428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.448625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-25ab0637-7e16-4c54-b0e3-3b3a7fbfd428 None None] MFA Rules not processed for user `9d717fc739eb4a1fb84e53a06b5e9ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:09.449861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-25ab0637-7e16-4c54-b0e3-3b3a7fbfd428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:09.513588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1070/2149] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:09.518590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82677081-60c9-4843-ad31-88f03b050a4f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:09.579172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82677081-60c9-4843-ad31-88f03b050a4f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:09.584278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265dcb6-2650-4b00-8f3b-ca87c53ffd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:09.584969 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265dcb6-2650-4b00-8f3b-ca87c53ffd1e admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.585261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265dcb6-2650-4b00-8f3b-ca87c53ffd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.585546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265dcb6-2650-4b00-8f3b-ca87c53ffd1e admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.586258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265dcb6-2650-4b00-8f3b-ca87c53ffd1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:09.588169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265dcb6-2650-4b00-8f3b-ca87c53ffd1e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:09.614968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1080/2150] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:09 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:09.619160 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab0637-7e16-4c54-b0e3-3b3a7fbfd428 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:09.630621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab0637-7e16-4c54-b0e3-3b3a7fbfd428 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:09.675316 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:09.688090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91574b4-704f-4758-b605-98392a51e192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:09.688810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91574b4-704f-4758-b605-98392a51e192 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.689092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91574b4-704f-4758-b605-98392a51e192 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.689368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91574b4-704f-4758-b605-98392a51e192 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.690066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91574b4-704f-4758-b605-98392a51e192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:09.691981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91574b4-704f-4758-b605-98392a51e192 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:09.712545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1071/2151] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:09 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:09.716404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265dcb6-2650-4b00-8f3b-ca87c53ffd1e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:09.726468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265dcb6-2650-4b00-8f3b-ca87c53ffd1e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:09.731057 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881eb977-7751-467b-b9f5-b7e804f6ba38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:09.731705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881eb977-7751-467b-b9f5-b7e804f6ba38 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.731990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881eb977-7751-467b-b9f5-b7e804f6ba38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.732269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881eb977-7751-467b-b9f5-b7e804f6ba38 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.733036 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881eb977-7751-467b-b9f5-b7e804f6ba38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:09.734804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881eb977-7751-467b-b9f5-b7e804f6ba38 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:09.746407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1081/2152] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:09.750848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91574b4-704f-4758-b605-98392a51e192 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:09.761042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91574b4-704f-4758-b605-98392a51e192 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:09.766185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b2122-35f5-4d1e-9fac-c71f62edb678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:09.767052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b2122-35f5-4d1e-9fac-c71f62edb678 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.767334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b2122-35f5-4d1e-9fac-c71f62edb678 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.767609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b2122-35f5-4d1e-9fac-c71f62edb678 admin admin] PATH_INFO: `/v3/projects/d1b7f615f45244c489400889ce931735/users/1a1392f0898f4afa966976f912a9b74b/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.782215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b2122-35f5-4d1e-9fac-c71f62edb678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1b7f615f45244c489400889ce931735, user_id=1a1392f0898f4afa966976f912a9b74b, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:09.784866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b2122-35f5-4d1e-9fac-c71f62edb678 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:09.797815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1072/2153] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:09 2022] PUT /identity/v3/projects/d1b7f615f45244c489400889ce931735/users/1a1392f0898f4afa966976f912a9b74b/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:48:09.801637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881eb977-7751-467b-b9f5-b7e804f6ba38 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:09.811949 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881eb977-7751-467b-b9f5-b7e804f6ba38 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:09.815796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:09.875719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54129ee-1af5-48ef-8490-3192c3d6541f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:09.876397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54129ee-1af5-48ef-8490-3192c3d6541f admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.876746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54129ee-1af5-48ef-8490-3192c3d6541f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.877029 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54129ee-1af5-48ef-8490-3192c3d6541f admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.877762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54129ee-1af5-48ef-8490-3192c3d6541f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:09.880502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54129ee-1af5-48ef-8490-3192c3d6541f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:09.899258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1082/2154] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:09.900647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b2122-35f5-4d1e-9fac-c71f62edb678 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:09.914913 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b2122-35f5-4d1e-9fac-c71f62edb678 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:09.935554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1058c7-b4b0-4042-a1a8-1a855a6c9368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:09.941143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1058c7-b4b0-4042-a1a8-1a855a6c9368 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.941456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1058c7-b4b0-4042-a1a8-1a855a6c9368 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.941729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1058c7-b4b0-4042-a1a8-1a855a6c9368 admin admin] PATH_INFO: `/v3/projects/d1b7f615f45244c489400889ce931735/users/1a1392f0898f4afa966976f912a9b74b/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:09.946503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1058c7-b4b0-4042-a1a8-1a855a6c9368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1b7f615f45244c489400889ce931735, user_id=1a1392f0898f4afa966976f912a9b74b, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:09.949443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1058c7-b4b0-4042-a1a8-1a855a6c9368 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:09.964121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1073/2155] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:09 2022] PUT /identity/v3/projects/d1b7f615f45244c489400889ce931735/users/1a1392f0898f4afa966976f912a9b74b/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:09.974100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b41b1-2d35-4aaa-8413-9ea87ab9437a None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:09.974393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b41b1-2d35-4aaa-8413-9ea87ab9437a None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:09.974670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b41b1-2d35-4aaa-8413-9ea87ab9437a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:10.047055 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-913b41b1-2d35-4aaa-8413-9ea87ab9437a None None] MFA Rules not processed for user `1a1392f0898f4afa966976f912a9b74b`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:10.048885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-913b41b1-2d35-4aaa-8413-9ea87ab9437a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:10.177311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1083/2156] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:10.184186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1058c7-b4b0-4042-a1a8-1a855a6c9368 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:10.186705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8b1058c7-b4b0-4042-a1a8-1a855a6c9368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:10.286254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1058c7-b4b0-4042-a1a8-1a855a6c9368 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:10.290128 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0ec8d-32e0-4986-b8d1-b32bff831bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:10.290785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0ec8d-32e0-4986-b8d1-b32bff831bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:10.291068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0ec8d-32e0-4986-b8d1-b32bff831bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:10.291352 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0ec8d-32e0-4986-b8d1-b32bff831bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:10.357496 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0ec8d-32e0-4986-b8d1-b32bff831bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:10.359761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0ec8d-32e0-4986-b8d1-b32bff831bb8 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:10.372804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1074/2157] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:11.144950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b41b1-2d35-4aaa-8413-9ea87ab9437a None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:11.146381 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-913b41b1-2d35-4aaa-8413-9ea87ab9437a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:11.218242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b41b1-2d35-4aaa-8413-9ea87ab9437a None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:11.224085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:11.225009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:11.225296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:11.225578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] PATH_INFO: `/v3/users/b891856026ac42fa8669e6ff451d184b` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:11.226925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b891856026ac42fa8669e6ff451d184b)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:11.228876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:11.283723 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:11.305775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b891856026ac42fa8669e6ff451d184b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:11.306530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b891856026ac42fa8669e6ff451d184b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:11.318012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b891856026ac42fa8669e6ff451d184b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:11.333265 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b891856026ac42fa8669e6ff451d184b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:11.347512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b891856026ac42fa8669e6ff451d184b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:11.361367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756fea1-dc92-40cb-af5f-e79655c351f5 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:11.361941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756fea1-dc92-40cb-af5f-e79655c351f5 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:11.361941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756fea1-dc92-40cb-af5f-e79655c351f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:11.364758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1084/2158] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:11 2022] DELETE /identity/v3/users/b891856026ac42fa8669e6ff451d184b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:11.384421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5756fea1-dc92-40cb-af5f-e79655c351f5 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:11.398690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5756fea1-dc92-40cb-af5f-e79655c351f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:11.466238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:11.481296 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1075/2159] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:11.526280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:11.528502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:11.605669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c0578-24e4-4ab6-a7e0-e18671f21616 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:11.610483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e96a2b-3199-4995-a730-bc5368c9ae38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:11.611162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e96a2b-3199-4995-a730-bc5368c9ae38 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:11.611965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e96a2b-3199-4995-a730-bc5368c9ae38 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:11.612260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e96a2b-3199-4995-a730-bc5368c9ae38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:11.624294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e96a2b-3199-4995-a730-bc5368c9ae38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:11.625790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e96a2b-3199-4995-a730-bc5368c9ae38 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:11.641777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1085/2160] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:11.988416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756fea1-dc92-40cb-af5f-e79655c351f5 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:11.989993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5756fea1-dc92-40cb-af5f-e79655c351f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:12.083638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756fea1-dc92-40cb-af5f-e79655c351f5 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:12.089544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bac28-4a52-4d0d-be68-38bbf49dd1d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:12.090237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bac28-4a52-4d0d-be68-38bbf49dd1d6 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:12.090525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bac28-4a52-4d0d-be68-38bbf49dd1d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:12.090808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bac28-4a52-4d0d-be68-38bbf49dd1d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:12.094434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8a4bac28-4a52-4d0d-be68-38bbf49dd1d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:12.192541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bac28-4a52-4d0d-be68-38bbf49dd1d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:12.194831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bac28-4a52-4d0d-be68-38bbf49dd1d6 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:12.208015 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1076/2161] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:12.329076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e96a2b-3199-4995-a730-bc5368c9ae38 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:12.331100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-27e96a2b-3199-4995-a730-bc5368c9ae38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:12.359588 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e96a2b-3199-4995-a730-bc5368c9ae38 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:12.384108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:12.385042 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:12.385512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:12.385801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] PATH_INFO: `/v3/projects/d6a2c92dabea4b089aeaa684e1d97ed7` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:12.387038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6a2c92dabea4b089aeaa684e1d97ed7)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:12.390114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:12.510474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6a2c92dabea4b089aeaa684e1d97ed7', 'request_id': 'req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:12.521810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6a2c92dabea4b089aeaa684e1d97ed7', 'request_id': 'req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:12.529392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6a2c92dabea4b089aeaa684e1d97ed7', 'request_id': 'req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:12.544127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] The token cache is being invalidate because project d6a2c92dabea4b089aeaa684e1d97ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:12.544462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:12.545443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:12.547773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1086/2162] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:12 2022] DELETE /identity/v3/projects/d6a2c92dabea4b089aeaa684e1d97ed7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:12.552252 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bac28-4a52-4d0d-be68-38bbf49dd1d6 service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:12.569701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bac28-4a52-4d0d-be68-38bbf49dd1d6 service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:12.709940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:12.710852 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:12.711135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:12.711587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] PATH_INFO: `/v3/users/95a91ebc6538464893b20b55295c7240` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:12.715508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] RBAC: Authorizing `identity:delete_user(user_id=95a91ebc6538464893b20b55295c7240)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:12.717592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:12.866191 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95a91ebc6538464893b20b55295c7240'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:12.875080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:12.881452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95a91ebc6538464893b20b55295c7240'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:12.910646 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95a91ebc6538464893b20b55295c7240'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:12.918481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95a91ebc6538464893b20b55295c7240'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:12.919337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95a91ebc6538464893b20b55295c7240'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:12.929119 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1077/2163] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:12 2022] DELETE /identity/v3/users/95a91ebc6538464893b20b55295c7240 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:13.320393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:13.322031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:13.382574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:13.441139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9cb29-8478-44de-8ff4-971c9fc8b24c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:13.470811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:13.472286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:13.558100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4534d-2e61-44f6-85f4-0cc0dcdc4953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:13.559079 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4534d-2e61-44f6-85f4-0cc0dcdc4953 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:13.563780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4534d-2e61-44f6-85f4-0cc0dcdc4953 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:13.564262 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4534d-2e61-44f6-85f4-0cc0dcdc4953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:13.566679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-90e4534d-2e61-44f6-85f4-0cc0dcdc4953 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:13.608344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99398b4-8f6b-4838-a843-493b75b2347b admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:13.625309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c2423-ef10-4765-8567-c0bdb98bd6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:13.626748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c2423-ef10-4765-8567-c0bdb98bd6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:13.627055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c2423-ef10-4765-8567-c0bdb98bd6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:13.627424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c2423-ef10-4765-8567-c0bdb98bd6bc admin admin] PATH_INFO: `/v3/projects/d9f95b39da8a4461b302bbc6221d23ef` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:13.628973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c2423-ef10-4765-8567-c0bdb98bd6bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f95b39da8a4461b302bbc6221d23ef)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:13.631618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c2423-ef10-4765-8567-c0bdb98bd6bc admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:13.667433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4534d-2e61-44f6-85f4-0cc0dcdc4953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:13.668996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4534d-2e61-44f6-85f4-0cc0dcdc4953 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:13.681946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1087/2164] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:13.691242 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-227c2423-ef10-4765-8567-c0bdb98bd6bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f95b39da8a4461b302bbc6221d23ef', 'request_id': 'req-227c2423-ef10-4765-8567-c0bdb98bd6bc'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:13.699815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-227c2423-ef10-4765-8567-c0bdb98bd6bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f95b39da8a4461b302bbc6221d23ef', 'request_id': 'req-227c2423-ef10-4765-8567-c0bdb98bd6bc'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:13.707797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-227c2423-ef10-4765-8567-c0bdb98bd6bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f95b39da8a4461b302bbc6221d23ef', 'request_id': 'req-227c2423-ef10-4765-8567-c0bdb98bd6bc'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:13.723758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-227c2423-ef10-4765-8567-c0bdb98bd6bc admin admin] The token cache is being invalidate because project d9f95b39da8a4461b302bbc6221d23ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:13.724349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-227c2423-ef10-4765-8567-c0bdb98bd6bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:13.725188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-227c2423-ef10-4765-8567-c0bdb98bd6bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:13.727622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1078/2165] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:13 2022] DELETE /identity/v3/projects/d9f95b39da8a4461b302bbc6221d23ef => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:48:13.733323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80b003-9fee-4da4-8687-46088dc7695b None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:13.733761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80b003-9fee-4da4-8687-46088dc7695b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:13.734129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80b003-9fee-4da4-8687-46088dc7695b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:13.755269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-5a80b003-9fee-4da4-8687-46088dc7695b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:13.756541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-5a80b003-9fee-4da4-8687-46088dc7695b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:13.824994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1088/2166] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:13.833653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e86023-cb67-4e7e-89f4-d30cec131f6a None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:13.834142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e86023-cb67-4e7e-89f4-d30cec131f6a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:13.834428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e86023-cb67-4e7e-89f4-d30cec131f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:13.854601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-52e86023-cb67-4e7e-89f4-d30cec131f6a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:13.855868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-52e86023-cb67-4e7e-89f4-d30cec131f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:13.864056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1079/2167] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:13.868201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80b003-9fee-4da4-8687-46088dc7695b None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:13.877816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80b003-9fee-4da4-8687-46088dc7695b None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:13.882586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba426e-28e8-4c61-886f-6b6336e1143a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:13.883498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba426e-28e8-4c61-886f-6b6336e1143a admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:13.883781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba426e-28e8-4c61-886f-6b6336e1143a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:13.884060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba426e-28e8-4c61-886f-6b6336e1143a admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:13.884829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba426e-28e8-4c61-886f-6b6336e1143a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:48:13.885142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba426e-28e8-4c61-886f-6b6336e1143a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:13.886785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba426e-28e8-4c61-886f-6b6336e1143a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:13.900477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1089/2168] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:48:13.905745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e86023-cb67-4e7e-89f4-d30cec131f6a None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:13.916717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e86023-cb67-4e7e-89f4-d30cec131f6a None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:13.922193 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d79c1c-1432-4830-aade-22a8c6125c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:13.922883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d79c1c-1432-4830-aade-22a8c6125c04 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:13.923191 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d79c1c-1432-4830-aade-22a8c6125c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:13.923452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d79c1c-1432-4830-aade-22a8c6125c04 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:13.924183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d79c1c-1432-4830-aade-22a8c6125c04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:13.926834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d79c1c-1432-4830-aade-22a8c6125c04 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:13.961472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1080/2169] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:13 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:48:13.965904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba426e-28e8-4c61-886f-6b6336e1143a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:13.977948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba426e-28e8-4c61-886f-6b6336e1143a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:14.007423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:14.065696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e66c84-75fc-4b44-9831-353d0f477490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:14.067715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e66c84-75fc-4b44-9831-353d0f477490 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:14.068008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e66c84-75fc-4b44-9831-353d0f477490 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:14.068367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e66c84-75fc-4b44-9831-353d0f477490 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:14.069893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e66c84-75fc-4b44-9831-353d0f477490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:14.072195 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e66c84-75fc-4b44-9831-353d0f477490 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:14.110673 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1090/2170] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:13 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:48:14.112881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d79c1c-1432-4830-aade-22a8c6125c04 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:14.128909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d79c1c-1432-4830-aade-22a8c6125c04 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:14.135212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ca878-d489-4686-9a38-979397f5e35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:14.136216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ca878-d489-4686-9a38-979397f5e35f admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:14.136602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ca878-d489-4686-9a38-979397f5e35f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:14.136958 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ca878-d489-4686-9a38-979397f5e35f admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:14.137806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ca878-d489-4686-9a38-979397f5e35f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:14.140168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ca878-d489-4686-9a38-979397f5e35f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:14.159318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1081/2171] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:14.163579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e66c84-75fc-4b44-9831-353d0f477490 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:14.177038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e66c84-75fc-4b44-9831-353d0f477490 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:14.182689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41d7bc-5a10-4040-bcf4-1d05e93bd556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:14.183512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41d7bc-5a10-4040-bcf4-1d05e93bd556 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:14.183784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41d7bc-5a10-4040-bcf4-1d05e93bd556 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:14.184454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41d7bc-5a10-4040-bcf4-1d05e93bd556 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:14.184881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41d7bc-5a10-4040-bcf4-1d05e93bd556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:14.187149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41d7bc-5a10-4040-bcf4-1d05e93bd556 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:14.204219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1091/2172] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:14.205961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ca878-d489-4686-9a38-979397f5e35f admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:14.224394 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ca878-d489-4686-9a38-979397f5e35f admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:14.230533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744518c8-ed9e-4848-869a-cbe5acd8cd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:14.231754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744518c8-ed9e-4848-869a-cbe5acd8cd68 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:14.232300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744518c8-ed9e-4848-869a-cbe5acd8cd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:14.232837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744518c8-ed9e-4848-869a-cbe5acd8cd68 admin admin] PATH_INFO: `/v3/projects/97103ac8798c4ea8933e01881e33beaa/users/2bbd64c115f04c59b135bcb5865b54c9/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:14.264793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744518c8-ed9e-4848-869a-cbe5acd8cd68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97103ac8798c4ea8933e01881e33beaa, user_id=2bbd64c115f04c59b135bcb5865b54c9, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:14.267671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744518c8-ed9e-4848-869a-cbe5acd8cd68 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:14.288314 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:14.329213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1082/2173] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:14 2022] PUT /identity/v3/projects/97103ac8798c4ea8933e01881e33beaa/users/2bbd64c115f04c59b135bcb5865b54c9/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:14.330969 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6377f2-917f-472a-b9e1-82fba5b8c6a4 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:14.331272 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6377f2-917f-472a-b9e1-82fba5b8c6a4 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:14.331617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6377f2-917f-472a-b9e1-82fba5b8c6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:14.363842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-ed6377f2-917f-472a-b9e1-82fba5b8c6a4 None None] MFA Rules not processed for user `2bbd64c115f04c59b135bcb5865b54c9`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:14.365267 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ed6377f2-917f-472a-b9e1-82fba5b8c6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:14.473818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1092/2174] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:14.478094 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744518c8-ed9e-4848-869a-cbe5acd8cd68 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:14.581740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744518c8-ed9e-4848-869a-cbe5acd8cd68 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:14.586973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b57ddb-5a29-4734-8b2a-7526936c61b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:14.587658 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b57ddb-5a29-4734-8b2a-7526936c61b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:14.587952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b57ddb-5a29-4734-8b2a-7526936c61b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:14.588228 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b57ddb-5a29-4734-8b2a-7526936c61b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:14.588976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b57ddb-5a29-4734-8b2a-7526936c61b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:14.591148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b57ddb-5a29-4734-8b2a-7526936c61b4 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:14.631679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1083/2175] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:14 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:14.635268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6377f2-917f-472a-b9e1-82fba5b8c6a4 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:14.655569 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6377f2-917f-472a-b9e1-82fba5b8c6a4 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:14.766574 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6ab53-d10e-4634-be47-ed99a39d504f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:14.767492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6ab53-d10e-4634-be47-ed99a39d504f admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:14.768521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6ab53-d10e-4634-be47-ed99a39d504f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:14.768843 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6ab53-d10e-4634-be47-ed99a39d504f admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:14.769835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6ab53-d10e-4634-be47-ed99a39d504f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:14.772333 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6ab53-d10e-4634-be47-ed99a39d504f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:14.804249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b57ddb-5a29-4734-8b2a-7526936c61b4 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:14.806915 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-17b57ddb-5a29-4734-8b2a-7526936c61b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:14.810708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1093/2176] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:14 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:14.812332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6ab53-d10e-4634-be47-ed99a39d504f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:14.832612 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6ab53-d10e-4634-be47-ed99a39d504f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:14.848446 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c8d32-56d1-4754-9942-2ab73e7a8b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:14.849557 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c8d32-56d1-4754-9942-2ab73e7a8b04 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:14.850512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c8d32-56d1-4754-9942-2ab73e7a8b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:14.851002 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c8d32-56d1-4754-9942-2ab73e7a8b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:14.852008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c8d32-56d1-4754-9942-2ab73e7a8b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:14.853925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c8d32-56d1-4754-9942-2ab73e7a8b04 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:14.857336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b57ddb-5a29-4734-8b2a-7526936c61b4 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:14.874755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1094/2177] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:14.883184 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c8d32-56d1-4754-9942-2ab73e7a8b04 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:14.885004 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:14.886597 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:14.887127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:14.887513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] PATH_INFO: `/v3/users/bc39f30010e94eb5862285347575f673` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:14.890666 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc39f30010e94eb5862285347575f673)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:14.893046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:14.913912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c8d32-56d1-4754-9942-2ab73e7a8b04 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:14.929706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b07574-f979-40e7-93a3-54fe24a2cfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:14.937737 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b07574-f979-40e7-93a3-54fe24a2cfd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:14.940542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b07574-f979-40e7-93a3-54fe24a2cfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:14.941364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b07574-f979-40e7-93a3-54fe24a2cfd9 admin admin] PATH_INFO: `/v3/projects/94e59ee441674fae9cd4997c5f14bfeb/users/4daec1190fdb4e5f802d4393b18fc75d/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:14.966055 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:14.989146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b07574-f979-40e7-93a3-54fe24a2cfd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e59ee441674fae9cd4997c5f14bfeb, user_id=4daec1190fdb4e5f802d4393b18fc75d, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:14.992138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b07574-f979-40e7-93a3-54fe24a2cfd9 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:15.007050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc39f30010e94eb5862285347575f673'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:15.024927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc39f30010e94eb5862285347575f673'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:15.026260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1095/2178] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:14 2022] PUT /identity/v3/projects/94e59ee441674fae9cd4997c5f14bfeb/users/4daec1190fdb4e5f802d4393b18fc75d/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:15.039138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b07574-f979-40e7-93a3-54fe24a2cfd9 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:15.060707 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc39f30010e94eb5862285347575f673'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:15.078375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc39f30010e94eb5862285347575f673'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:15.079745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc39f30010e94eb5862285347575f673'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:15.090426 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1084/2179] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:14 2022] DELETE /identity/v3/users/bc39f30010e94eb5862285347575f673 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:15.151122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b07574-f979-40e7-93a3-54fe24a2cfd9 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:15.236230 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e2554-225d-4bf4-a3e5-b0c35ebca6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:15.237275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e2554-225d-4bf4-a3e5-b0c35ebca6fa admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:15.238233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e2554-225d-4bf4-a3e5-b0c35ebca6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:15.238535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e2554-225d-4bf4-a3e5-b0c35ebca6fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:15.239341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e2554-225d-4bf4-a3e5-b0c35ebca6fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:15.241203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e2554-225d-4bf4-a3e5-b0c35ebca6fa admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:15.256914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1096/2180] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:48:15.258648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:15.265561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:15.285674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dff72-8757-4971-847d-786616fcb273 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:15.293326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59d2d9-b8ae-4d98-8bae-d5fa63510fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:15.294052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59d2d9-b8ae-4d98-8bae-d5fa63510fce admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:15.294341 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59d2d9-b8ae-4d98-8bae-d5fa63510fce admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:15.294685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59d2d9-b8ae-4d98-8bae-d5fa63510fce admin admin] PATH_INFO: `/v3/projects/94e59ee441674fae9cd4997c5f14bfeb/users/4daec1190fdb4e5f802d4393b18fc75d/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:15.297228 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59d2d9-b8ae-4d98-8bae-d5fa63510fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e59ee441674fae9cd4997c5f14bfeb, user_id=4daec1190fdb4e5f802d4393b18fc75d, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:15.299987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59d2d9-b8ae-4d98-8bae-d5fa63510fce admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:15.320689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1085/2181] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:15 2022] PUT /identity/v3/projects/94e59ee441674fae9cd4997c5f14bfeb/users/4daec1190fdb4e5f802d4393b18fc75d/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:15.326159 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34704e2d-6468-4506-b4aa-2ef045a969ec None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:15.327380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34704e2d-6468-4506-b4aa-2ef045a969ec None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:15.327676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34704e2d-6468-4506-b4aa-2ef045a969ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:15.363864 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59d2d9-b8ae-4d98-8bae-d5fa63510fce admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:15.366023 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-fe59d2d9-b8ae-4d98-8bae-d5fa63510fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:15.379155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-34704e2d-6468-4506-b4aa-2ef045a969ec None None] MFA Rules not processed for user `4daec1190fdb4e5f802d4393b18fc75d`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:15.380488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-34704e2d-6468-4506-b4aa-2ef045a969ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:15.429239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:15.454528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59d2d9-b8ae-4d98-8bae-d5fa63510fce admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:15.459400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36be79-e5c8-455c-8286-beea457eb7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:15.460273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36be79-e5c8-455c-8286-beea457eb7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:15.460556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36be79-e5c8-455c-8286-beea457eb7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:15.460862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36be79-e5c8-455c-8286-beea457eb7d4 admin admin] PATH_INFO: `/v3/projects/aa18cc0817c848a782e4176af753e3e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:15.462058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36be79-e5c8-455c-8286-beea457eb7d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa18cc0817c848a782e4176af753e3e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:15.463996 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36be79-e5c8-455c-8286-beea457eb7d4 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:15.469340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1097/2182] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:15.482724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34704e2d-6468-4506-b4aa-2ef045a969ec None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:15.484250 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-34704e2d-6468-4506-b4aa-2ef045a969ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:15.511066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:15.516351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0e36be79-e5c8-455c-8286-beea457eb7d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa18cc0817c848a782e4176af753e3e8', 'request_id': 'req-0e36be79-e5c8-455c-8286-beea457eb7d4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:15.526211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0e36be79-e5c8-455c-8286-beea457eb7d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa18cc0817c848a782e4176af753e3e8', 'request_id': 'req-0e36be79-e5c8-455c-8286-beea457eb7d4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:15.534111 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0e36be79-e5c8-455c-8286-beea457eb7d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa18cc0817c848a782e4176af753e3e8', 'request_id': 'req-0e36be79-e5c8-455c-8286-beea457eb7d4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:15.555474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0e36be79-e5c8-455c-8286-beea457eb7d4 admin admin] The token cache is being invalidate because project aa18cc0817c848a782e4176af753e3e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:15.555775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0e36be79-e5c8-455c-8286-beea457eb7d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:15.556464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0e36be79-e5c8-455c-8286-beea457eb7d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:15.561423 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1086/2183] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:15 2022] DELETE /identity/v3/projects/aa18cc0817c848a782e4176af753e3e8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:15.563293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87d05c-68e2-41bc-8eac-6d172e3458fa None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:15.563677 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87d05c-68e2-41bc-8eac-6d172e3458fa None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:15.563927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87d05c-68e2-41bc-8eac-6d172e3458fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:15.571340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34704e2d-6468-4506-b4aa-2ef045a969ec None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:15.587878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ab87d05c-68e2-41bc-8eac-6d172e3458fa None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:15.589177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ab87d05c-68e2-41bc-8eac-6d172e3458fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:15.640694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fffd81-3aba-45ff-9bd3-b0ed16535e21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:15.641807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fffd81-3aba-45ff-9bd3-b0ed16535e21 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:15.641807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fffd81-3aba-45ff-9bd3-b0ed16535e21 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:15.642015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fffd81-3aba-45ff-9bd3-b0ed16535e21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:15.643730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-85fffd81-3aba-45ff-9bd3-b0ed16535e21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:15.675388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1087/2184] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:15.683485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d1b77-9054-4f15-b0d5-6c8001f62106 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:15.683787 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d1b77-9054-4f15-b0d5-6c8001f62106 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:15.684076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d1b77-9054-4f15-b0d5-6c8001f62106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:15.706428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-2b2d1b77-9054-4f15-b0d5-6c8001f62106 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:15.707668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2b2d1b77-9054-4f15-b0d5-6c8001f62106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:15.716803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1088/2185] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:15.720946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d1b77-9054-4f15-b0d5-6c8001f62106 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:15.729614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fffd81-3aba-45ff-9bd3-b0ed16535e21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:15.730825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d1b77-9054-4f15-b0d5-6c8001f62106 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:15.731086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fffd81-3aba-45ff-9bd3-b0ed16535e21 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:15.735862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47040211-fff7-430b-a7fb-82b478c48934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:15.736737 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47040211-fff7-430b-a7fb-82b478c48934 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:15.737094 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47040211-fff7-430b-a7fb-82b478c48934 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:15.737443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47040211-fff7-430b-a7fb-82b478c48934 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:15.738206 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47040211-fff7-430b-a7fb-82b478c48934 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:48:15.738614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47040211-fff7-430b-a7fb-82b478c48934 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:15.740345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47040211-fff7-430b-a7fb-82b478c48934 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:15.744680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1098/2186] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:48:15.754782 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1089/2187] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:48:15.760752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fffd81-3aba-45ff-9bd3-b0ed16535e21 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:15.772600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fffd81-3aba-45ff-9bd3-b0ed16535e21 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:15.778010 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4722a-a4c6-4374-88ea-f130b48374f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:15.778617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4722a-a4c6-4374-88ea-f130b48374f4 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:15.778909 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4722a-a4c6-4374-88ea-f130b48374f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:15.779192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4722a-a4c6-4374-88ea-f130b48374f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:15.779970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4722a-a4c6-4374-88ea-f130b48374f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:15.781970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4722a-a4c6-4374-88ea-f130b48374f4 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:15.820665 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1099/2188] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:15 2022] POST /identity/v3/projects => generated 389 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:15.822770 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47040211-fff7-430b-a7fb-82b478c48934 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:15.833930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47040211-fff7-430b-a7fb-82b478c48934 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:15.892817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:15.905953 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c153958-da2b-4cce-8d07-3c2fa5dcb472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:15.907429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c153958-da2b-4cce-8d07-3c2fa5dcb472 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:15.908047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c153958-da2b-4cce-8d07-3c2fa5dcb472 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:15.908670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c153958-da2b-4cce-8d07-3c2fa5dcb472 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:15.909619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c153958-da2b-4cce-8d07-3c2fa5dcb472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:15.911911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c153958-da2b-4cce-8d07-3c2fa5dcb472 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:15.935196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1090/2189] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:15 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:15.940999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4722a-a4c6-4374-88ea-f130b48374f4 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:15.956186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4722a-a4c6-4374-88ea-f130b48374f4 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:15.961440 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9274e-fb33-4df9-992d-5a481618fd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:15.961930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9274e-fb33-4df9-992d-5a481618fd10 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:15.962225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9274e-fb33-4df9-992d-5a481618fd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:15.962504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9274e-fb33-4df9-992d-5a481618fd10 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:15.963252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9274e-fb33-4df9-992d-5a481618fd10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:15.965113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9274e-fb33-4df9-992d-5a481618fd10 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:15.980151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1100/2190] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:15.981392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c153958-da2b-4cce-8d07-3c2fa5dcb472 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:15.994607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c153958-da2b-4cce-8d07-3c2fa5dcb472 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:16.010427 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c830a03-93f9-4fb1-a54f-6aad452b8da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:16.011124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c830a03-93f9-4fb1-a54f-6aad452b8da7 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:16.011388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c830a03-93f9-4fb1-a54f-6aad452b8da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:16.011699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c830a03-93f9-4fb1-a54f-6aad452b8da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:16.012600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c830a03-93f9-4fb1-a54f-6aad452b8da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:16.014310 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c830a03-93f9-4fb1-a54f-6aad452b8da7 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:16.025990 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1091/2191] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:16.030993 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9274e-fb33-4df9-992d-5a481618fd10 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:16.041893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9274e-fb33-4df9-992d-5a481618fd10 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:16.047642 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918f01d-4133-40cf-b08b-30972a180cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:16.048515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918f01d-4133-40cf-b08b-30972a180cff admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:16.049904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918f01d-4133-40cf-b08b-30972a180cff admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:16.050191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918f01d-4133-40cf-b08b-30972a180cff admin admin] PATH_INFO: `/v3/projects/f3491b85dc4543c88f500828b2df1d2d/users/d3f0b30685e94f83883d682731644a89/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:16.067094 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918f01d-4133-40cf-b08b-30972a180cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3491b85dc4543c88f500828b2df1d2d, user_id=d3f0b30685e94f83883d682731644a89, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:16.069820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918f01d-4133-40cf-b08b-30972a180cff admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:16.082354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1101/2192] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:16 2022] PUT /identity/v3/projects/f3491b85dc4543c88f500828b2df1d2d/users/d3f0b30685e94f83883d682731644a89/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:16.087498 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf29c7-29d8-4ade-bfb3-61afe16ce5d6 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:16.087796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf29c7-29d8-4ade-bfb3-61afe16ce5d6 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:16.088082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf29c7-29d8-4ade-bfb3-61afe16ce5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:16.108141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-93bf29c7-29d8-4ade-bfb3-61afe16ce5d6 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:16.110000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-93bf29c7-29d8-4ade-bfb3-61afe16ce5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:16.129911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918f01d-4133-40cf-b08b-30972a180cff admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:16.160384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:16.182893 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1092/2193] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:16.190912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf29c7-29d8-4ade-bfb3-61afe16ce5d6 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:16.192536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-93bf29c7-29d8-4ade-bfb3-61afe16ce5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:16.206603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918f01d-4133-40cf-b08b-30972a180cff admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:16.213570 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1e4aa-4e05-4a4c-8570-007950533834 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4daec1190fdb4e5f802d4393b18fc75d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e59ee441674fae9cd4997c5f14bfeb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:16.214336 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1e4aa-4e05-4a4c-8570-007950533834 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:16.215067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1e4aa-4e05-4a4c-8570-007950533834 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:16.215435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1e4aa-4e05-4a4c-8570-007950533834 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:16.216316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1e4aa-4e05-4a4c-8570-007950533834 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:48:16.216715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1e4aa-4e05-4a4c-8570-007950533834 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:16.219132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1e4aa-4e05-4a4c-8570-007950533834 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:16.234687 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1102/2194] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:48:16.238827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1e4aa-4e05-4a4c-8570-007950533834 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:16.251664 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1e4aa-4e05-4a4c-8570-007950533834 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:16.257067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341cabc-20a8-41b9-956a-90763dca594b tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4daec1190fdb4e5f802d4393b18fc75d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e59ee441674fae9cd4997c5f14bfeb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:16.258216 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341cabc-20a8-41b9-956a-90763dca594b tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:16.258332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341cabc-20a8-41b9-956a-90763dca594b tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:16.258542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341cabc-20a8-41b9-956a-90763dca594b tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:16.259246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341cabc-20a8-41b9-956a-90763dca594b tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:16.261201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341cabc-20a8-41b9-956a-90763dca594b tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:16.261814 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf29c7-29d8-4ade-bfb3-61afe16ce5d6 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:16.265695 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a223585-0441-4884-8866-6bdc06ecc66c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:16.266345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a223585-0441-4884-8866-6bdc06ecc66c service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:16.266848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a223585-0441-4884-8866-6bdc06ecc66c service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:16.267316 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a223585-0441-4884-8866-6bdc06ecc66c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:16.279503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a223585-0441-4884-8866-6bdc06ecc66c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:16.281473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a223585-0441-4884-8866-6bdc06ecc66c service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:16.292947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1103/2195] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:16 2022] POST /identity/v3/projects => generated 355 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:48:16.303975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341cabc-20a8-41b9-956a-90763dca594b tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:16.308611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1093/2196] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:16.370060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341cabc-20a8-41b9-956a-90763dca594b tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:16.376018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d9e34-10f3-4e37-a133-d07acaaaf451 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4daec1190fdb4e5f802d4393b18fc75d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e59ee441674fae9cd4997c5f14bfeb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:16.376865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d9e34-10f3-4e37-a133-d07acaaaf451 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:16.377161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d9e34-10f3-4e37-a133-d07acaaaf451 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:16.377485 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d9e34-10f3-4e37-a133-d07acaaaf451 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:16.378259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d9e34-10f3-4e37-a133-d07acaaaf451 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:48:16.378575 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d9e34-10f3-4e37-a133-d07acaaaf451 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:16.381139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d9e34-10f3-4e37-a133-d07acaaaf451 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:16.399707 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1104/2197] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:48:16.443789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a223585-0441-4884-8866-6bdc06ecc66c service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:16.456067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a223585-0441-4884-8866-6bdc06ecc66c service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:16.462741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496607e-6dae-434d-87e6-132cd827b006 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4daec1190fdb4e5f802d4393b18fc75d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e59ee441674fae9cd4997c5f14bfeb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:16.463633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496607e-6dae-434d-87e6-132cd827b006 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:16.463938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496607e-6dae-434d-87e6-132cd827b006 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:16.464237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496607e-6dae-434d-87e6-132cd827b006 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] PATH_INFO: `/v3/projects/94efc1c9d12b489ca35cc56d47bd1427` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:16.466094 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496607e-6dae-434d-87e6-132cd827b006 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: Authorizing `identity:delete_project(project_id=94efc1c9d12b489ca35cc56d47bd1427)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:16.469303 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496607e-6dae-434d-87e6-132cd827b006 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:16.540882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-5496607e-6dae-434d-87e6-132cd827b006 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94efc1c9d12b489ca35cc56d47bd1427', 'request_id': 'req-5496607e-6dae-434d-87e6-132cd827b006'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:16.551843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-5496607e-6dae-434d-87e6-132cd827b006 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94efc1c9d12b489ca35cc56d47bd1427', 'request_id': 'req-5496607e-6dae-434d-87e6-132cd827b006'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:16.560055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-5496607e-6dae-434d-87e6-132cd827b006 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94efc1c9d12b489ca35cc56d47bd1427', 'request_id': 'req-5496607e-6dae-434d-87e6-132cd827b006'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:16.562851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:16.574938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-5496607e-6dae-434d-87e6-132cd827b006 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] The token cache is being invalidate because project 94efc1c9d12b489ca35cc56d47bd1427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:16.575362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-5496607e-6dae-434d-87e6-132cd827b006 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:16.576253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-5496607e-6dae-434d-87e6-132cd827b006 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:16.581943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1094/2198] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:16 2022] DELETE /identity/v3/projects/94efc1c9d12b489ca35cc56d47bd1427 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:16.829160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d9e34-10f3-4e37-a133-d07acaaaf451 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:16.832794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-cb0d9e34-10f3-4e37-a133-d07acaaaf451 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:16.918392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d9e34-10f3-4e37-a133-d07acaaaf451 tempest-VolumeQuotaClassesTest-850941560 tempest-VolumeQuotaClassesTest-850941560-project-admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:16.956565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:16.976383 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:16.977316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:16.977600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:16.977882 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] PATH_INFO: `/v3/users/2bbd64c115f04c59b135bcb5865b54c9` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:16.979221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bbd64c115f04c59b135bcb5865b54c9)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:16.981478 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:17.051019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bbd64c115f04c59b135bcb5865b54c9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:17.051786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bbd64c115f04c59b135bcb5865b54c9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:17.073861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bbd64c115f04c59b135bcb5865b54c9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:17.091067 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bbd64c115f04c59b135bcb5865b54c9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:17.108678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bbd64c115f04c59b135bcb5865b54c9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:17.127913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1105/2199] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:16 2022] DELETE /identity/v3/users/2bbd64c115f04c59b135bcb5865b54c9 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:17.132317 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215eb1f-1fbb-4f61-a7a6-538436b4e58d None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:17.133098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215eb1f-1fbb-4f61-a7a6-538436b4e58d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:17.133209 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215eb1f-1fbb-4f61-a7a6-538436b4e58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:17.156045 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-8215eb1f-1fbb-4f61-a7a6-538436b4e58d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:17.157502 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8215eb1f-1fbb-4f61-a7a6-538436b4e58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:17.261090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1095/2200] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:17.278649 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:17.280737 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:17.348598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215eb1f-1fbb-4f61-a7a6-538436b4e58d None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:17.350939 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8215eb1f-1fbb-4f61-a7a6-538436b4e58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:17.365603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215eb1f-1fbb-4f61-a7a6-538436b4e58d None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:17.372489 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:17.374866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:17.375376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:17.375948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] PATH_INFO: `/v3/users/0cf75c04abc5478f9557908bd94afd9d` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:17.377728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf75c04abc5478f9557908bd94afd9d)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:17.380155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:17.419235 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8f13f-4b10-43d1-a3b4-62e760745c10 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:17.423594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33c648-5219-49c8-8585-44b1e2a98acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:17.424280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33c648-5219-49c8-8585-44b1e2a98acf service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:17.424660 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33c648-5219-49c8-8585-44b1e2a98acf service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:17.424933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33c648-5219-49c8-8585-44b1e2a98acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:17.440599 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33c648-5219-49c8-8585-44b1e2a98acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:17.443478 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33c648-5219-49c8-8585-44b1e2a98acf service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:17.455862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf75c04abc5478f9557908bd94afd9d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:17.463270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1106/2201] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:17.471040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf75c04abc5478f9557908bd94afd9d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:17.490382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf75c04abc5478f9557908bd94afd9d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:17.503060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf75c04abc5478f9557908bd94afd9d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:17.504508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf75c04abc5478f9557908bd94afd9d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:17.520123 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1096/2202] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:17 2022] DELETE /identity/v3/users/0cf75c04abc5478f9557908bd94afd9d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:17.521340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b110c36-f0ba-483c-947b-469ce1b2a295 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:17.522229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b110c36-f0ba-483c-947b-469ce1b2a295 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:17.522738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b110c36-f0ba-483c-947b-469ce1b2a295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:17.555935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-6b110c36-f0ba-483c-947b-469ce1b2a295 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:17.558374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6b110c36-f0ba-483c-947b-469ce1b2a295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:17.697836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1107/2203] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:48:17.724273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:17.726181 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:17.802177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e4cb1-5255-47ed-9ebf-410bd56a5f4e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:17.807301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be485b-39a3-4e23-b1dd-138460fc334f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:17.808213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be485b-39a3-4e23-b1dd-138460fc334f service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:17.809375 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be485b-39a3-4e23-b1dd-138460fc334f service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:17.809889 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be485b-39a3-4e23-b1dd-138460fc334f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:17.821550 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be485b-39a3-4e23-b1dd-138460fc334f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:17.823374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be485b-39a3-4e23-b1dd-138460fc334f service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:17.834582 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1097/2204] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:17.994499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b110c36-f0ba-483c-947b-469ce1b2a295 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:17.996434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6b110c36-f0ba-483c-947b-469ce1b2a295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:18.004688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:18.015447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b110c36-f0ba-483c-947b-469ce1b2a295 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:18.024945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:18.030518 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:18.030827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:18.032069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] PATH_INFO: `/v3/projects/97103ac8798c4ea8933e01881e33beaa` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:18.033579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=97103ac8798c4ea8933e01881e33beaa)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:18.040038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be485b-39a3-4e23-b1dd-138460fc334f service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:18.042927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e0be485b-39a3-4e23-b1dd-138460fc334f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:18.055994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:18.070145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be485b-39a3-4e23-b1dd-138460fc334f service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:18.101071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:18.101970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:18.102274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:18.102503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] PATH_INFO: `/v3/users/209152d018ef450e97ca5e6955db0009` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:18.108626 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=209152d018ef450e97ca5e6955db0009)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:18.113606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:18.128268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:18.238976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97103ac8798c4ea8933e01881e33beaa', 'request_id': 'req-09a4b165-8510-44ef-bfe2-a734cc4b3aba'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.261541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97103ac8798c4ea8933e01881e33beaa', 'request_id': 'req-09a4b165-8510-44ef-bfe2-a734cc4b3aba'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.267126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '209152d018ef450e97ca5e6955db0009'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.273910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97103ac8798c4ea8933e01881e33beaa', 'request_id': 'req-09a4b165-8510-44ef-bfe2-a734cc4b3aba'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.292811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '209152d018ef450e97ca5e6955db0009'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.305571 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] The token cache is being invalidate because project 97103ac8798c4ea8933e01881e33beaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:18.305571 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.305571 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.308096 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1108/2205] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:17 2022] DELETE /identity/v3/projects/97103ac8798c4ea8933e01881e33beaa => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:18.313240 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:18.330923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:18.343935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '209152d018ef450e97ca5e6955db0009'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.358400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '209152d018ef450e97ca5e6955db0009'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.359441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '209152d018ef450e97ca5e6955db0009'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.373861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1098/2206] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:18 2022] DELETE /identity/v3/users/209152d018ef450e97ca5e6955db0009 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:48:18.382563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:18.388642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:18.470754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4b165-8510-44ef-bfe2-a734cc4b3aba admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:18.537223 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69eb5d3-8cea-4985-aa58-b8ae7310d2ee admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:18.543769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24320a3e-f3e7-40d5-ac67-1d05d653ea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:18.546472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24320a3e-f3e7-40d5-ac67-1d05d653ea91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:18.547465 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24320a3e-f3e7-40d5-ac67-1d05d653ea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:18.548484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24320a3e-f3e7-40d5-ac67-1d05d653ea91 admin admin] PATH_INFO: `/v3/projects/778a06bd709a4e3abe691e5624ec6d4a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:18.555391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24320a3e-f3e7-40d5-ac67-1d05d653ea91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=778a06bd709a4e3abe691e5624ec6d4a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:18.559509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24320a3e-f3e7-40d5-ac67-1d05d653ea91 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:18.606641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b4780-10ad-4db6-84b7-fd123ccf3fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:18.607604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b4780-10ad-4db6-84b7-fd123ccf3fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:18.607896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b4780-10ad-4db6-84b7-fd123ccf3fae admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:18.608178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b4780-10ad-4db6-84b7-fd123ccf3fae admin admin] PATH_INFO: `/v3/users/4daec1190fdb4e5f802d4393b18fc75d` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:18.609513 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b4780-10ad-4db6-84b7-fd123ccf3fae admin admin] RBAC: Authorizing `identity:delete_user(user_id=4daec1190fdb4e5f802d4393b18fc75d)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:18.612786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b4780-10ad-4db6-84b7-fd123ccf3fae admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:18.690324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-24320a3e-f3e7-40d5-ac67-1d05d653ea91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778a06bd709a4e3abe691e5624ec6d4a', 'request_id': 'req-24320a3e-f3e7-40d5-ac67-1d05d653ea91'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.696773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:18.710492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-24320a3e-f3e7-40d5-ac67-1d05d653ea91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778a06bd709a4e3abe691e5624ec6d4a', 'request_id': 'req-24320a3e-f3e7-40d5-ac67-1d05d653ea91'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.712937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d85b4780-10ad-4db6-84b7-fd123ccf3fae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4daec1190fdb4e5f802d4393b18fc75d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.713884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d85b4780-10ad-4db6-84b7-fd123ccf3fae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4daec1190fdb4e5f802d4393b18fc75d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.718332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-24320a3e-f3e7-40d5-ac67-1d05d653ea91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778a06bd709a4e3abe691e5624ec6d4a', 'request_id': 'req-24320a3e-f3e7-40d5-ac67-1d05d653ea91'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.725533 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d85b4780-10ad-4db6-84b7-fd123ccf3fae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4daec1190fdb4e5f802d4393b18fc75d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.739606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-24320a3e-f3e7-40d5-ac67-1d05d653ea91 admin admin] The token cache is being invalidate because project 778a06bd709a4e3abe691e5624ec6d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:18.740329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-24320a3e-f3e7-40d5-ac67-1d05d653ea91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.741268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-24320a3e-f3e7-40d5-ac67-1d05d653ea91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.746270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d85b4780-10ad-4db6-84b7-fd123ccf3fae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4daec1190fdb4e5f802d4393b18fc75d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.753552 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d85b4780-10ad-4db6-84b7-fd123ccf3fae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4daec1190fdb4e5f802d4393b18fc75d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:18.759041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1099/2207] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:18 2022] DELETE /identity/v3/projects/778a06bd709a4e3abe691e5624ec6d4a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:18.760498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1109/2208] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:18 2022] DELETE /identity/v3/users/4daec1190fdb4e5f802d4393b18fc75d => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:18.765346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54242d2e-bc93-425d-aae6-992a7723c200 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:18.766950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab9ec2-7816-4166-9fb4-7f40601a1a30 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:18.767342 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54242d2e-bc93-425d-aae6-992a7723c200 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:18.767725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54242d2e-bc93-425d-aae6-992a7723c200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:18.769569 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab9ec2-7816-4166-9fb4-7f40601a1a30 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:18.771154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab9ec2-7816-4166-9fb4-7f40601a1a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:18.806202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-54242d2e-bc93-425d-aae6-992a7723c200 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:18.807535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-54242d2e-bc93-425d-aae6-992a7723c200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:18.818668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:18.819797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-fdab9ec2-7816-4166-9fb4-7f40601a1a30 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:18.822723 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fdab9ec2-7816-4166-9fb4-7f40601a1a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:18.893139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1100/2209] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:18.903310 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54242d2e-bc93-425d-aae6-992a7723c200 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:18.905218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-54242d2e-bc93-425d-aae6-992a7723c200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:18.944559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1110/2210] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:18.955289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c56e6a-4dc5-497a-beb8-60193d1703bd None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:18.955628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c56e6a-4dc5-497a-beb8-60193d1703bd None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:18.956149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c56e6a-4dc5-497a-beb8-60193d1703bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:18.979577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d1c56e6a-4dc5-497a-beb8-60193d1703bd None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:18.981603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d1c56e6a-4dc5-497a-beb8-60193d1703bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:19.002140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54242d2e-bc93-425d-aae6-992a7723c200 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:19.003189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1111/2211] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:19.007421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c56e6a-4dc5-497a-beb8-60193d1703bd None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:19.011615 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1e15a-f603-4299-97c8-850059c4fe67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:19.013566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1e15a-f603-4299-97c8-850059c4fe67 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:19.013911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1e15a-f603-4299-97c8-850059c4fe67 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:19.014240 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1e15a-f603-4299-97c8-850059c4fe67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:19.026410 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c56e6a-4dc5-497a-beb8-60193d1703bd None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:19.026639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1e15a-f603-4299-97c8-850059c4fe67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:19.028117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1e15a-f603-4299-97c8-850059c4fe67 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:19.044264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e481f-a992-400e-9871-250605f6d22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:19.045536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e481f-a992-400e-9871-250605f6d22a admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:19.045919 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e481f-a992-400e-9871-250605f6d22a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:19.046300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e481f-a992-400e-9871-250605f6d22a admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:19.047094 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e481f-a992-400e-9871-250605f6d22a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:48:19.047404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e481f-a992-400e-9871-250605f6d22a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:19.049168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e481f-a992-400e-9871-250605f6d22a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:19.055425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1101/2212] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:48:19.075094 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1112/2213] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:48:19.076674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1e15a-f603-4299-97c8-850059c4fe67 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:19.093376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1e15a-f603-4299-97c8-850059c4fe67 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:19.107374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94d654-eef9-4b9a-8d39-f5393321a68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:19.108985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94d654-eef9-4b9a-8d39-f5393321a68a admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:19.109465 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94d654-eef9-4b9a-8d39-f5393321a68a admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:19.110192 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94d654-eef9-4b9a-8d39-f5393321a68a admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:19.111044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94d654-eef9-4b9a-8d39-f5393321a68a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:19.113652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94d654-eef9-4b9a-8d39-f5393321a68a admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:19.157786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1102/2214] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:19 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:48:19.162250 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e481f-a992-400e-9871-250605f6d22a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:19.176905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e481f-a992-400e-9871-250605f6d22a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:19.304857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1c327-7f21-43da-9669-124966fc3545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:19.306299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1c327-7f21-43da-9669-124966fc3545 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:19.306421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1c327-7f21-43da-9669-124966fc3545 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:19.306708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1c327-7f21-43da-9669-124966fc3545 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:19.309917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1c327-7f21-43da-9669-124966fc3545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:19.317266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1c327-7f21-43da-9669-124966fc3545 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:19.355708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1113/2215] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:19 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:19.360009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94d654-eef9-4b9a-8d39-f5393321a68a admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:19.374750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94d654-eef9-4b9a-8d39-f5393321a68a admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:19.389445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad0f2f-d6a4-47af-9d8e-3ceaa65816a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:19.390533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad0f2f-d6a4-47af-9d8e-3ceaa65816a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:19.390931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad0f2f-d6a4-47af-9d8e-3ceaa65816a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:19.391273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad0f2f-d6a4-47af-9d8e-3ceaa65816a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:19.393225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad0f2f-d6a4-47af-9d8e-3ceaa65816a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:19.396231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad0f2f-d6a4-47af-9d8e-3ceaa65816a2 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:19.441964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1c327-7f21-43da-9669-124966fc3545 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:19.450371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1103/2216] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:19.453721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad0f2f-d6a4-47af-9d8e-3ceaa65816a2 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:19.462593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ecad0f2f-d6a4-47af-9d8e-3ceaa65816a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:19.472197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1c327-7f21-43da-9669-124966fc3545 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:19.483604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a9156-30f7-49cf-bc30-679df2e2c948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:19.484491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a9156-30f7-49cf-bc30-679df2e2c948 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:19.484883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a9156-30f7-49cf-bc30-679df2e2c948 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:19.486988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a9156-30f7-49cf-bc30-679df2e2c948 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:19.487766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a9156-30f7-49cf-bc30-679df2e2c948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:19.490406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a9156-30f7-49cf-bc30-679df2e2c948 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:19.498609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad0f2f-d6a4-47af-9d8e-3ceaa65816a2 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:19.508792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1114/2217] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:19.509657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a9156-30f7-49cf-bc30-679df2e2c948 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:19.525595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a9156-30f7-49cf-bc30-679df2e2c948 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:19.527871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:19.528973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:19.530427 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:19.531101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] PATH_INFO: `/v3/projects/94e59ee441674fae9cd4997c5f14bfeb` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:19.533089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1841e85-eda2-407f-b38c-8149d440bf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:19.533452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94e59ee441674fae9cd4997c5f14bfeb)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:19.534021 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1841e85-eda2-407f-b38c-8149d440bf83 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:19.534357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1841e85-eda2-407f-b38c-8149d440bf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:19.534687 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1841e85-eda2-407f-b38c-8149d440bf83 admin admin] PATH_INFO: `/v3/projects/3404b1a99987401183c8e3e851b382db/users/cd036c73e5c84fa1ba548be4eae80804/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:19.536808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:19.545118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:19.574475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1841e85-eda2-407f-b38c-8149d440bf83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3404b1a99987401183c8e3e851b382db, user_id=cd036c73e5c84fa1ba548be4eae80804, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:19.582325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1841e85-eda2-407f-b38c-8149d440bf83 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:19.608896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1115/2218] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:19 2022] PUT /identity/v3/projects/3404b1a99987401183c8e3e851b382db/users/cd036c73e5c84fa1ba548be4eae80804/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:19.613412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a881f23-e3cb-4415-b2a6-b0e5cd591511 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:19.614225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a881f23-e3cb-4415-b2a6-b0e5cd591511 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:19.614225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a881f23-e3cb-4415-b2a6-b0e5cd591511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:19.658422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7a881f23-e3cb-4415-b2a6-b0e5cd591511 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:19.659908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7a881f23-e3cb-4415-b2a6-b0e5cd591511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:19.665065 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94e59ee441674fae9cd4997c5f14bfeb', 'request_id': 'req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:19.678866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94e59ee441674fae9cd4997c5f14bfeb', 'request_id': 'req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:19.722413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94e59ee441674fae9cd4997c5f14bfeb', 'request_id': 'req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:19.749555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] The token cache is being invalidate because project 94e59ee441674fae9cd4997c5f14bfeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:19.749922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:19.750986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:19.753014 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1104/2219] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:19 2022] DELETE /identity/v3/projects/94e59ee441674fae9cd4997c5f14bfeb => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:19.753827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:19.755503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:19.794564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:19.823549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1116/2220] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:19.827510 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63f8cf-7340-4e69-9c2f-a877053fa3ce None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:19.827818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63f8cf-7340-4e69-9c2f-a877053fa3ce None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:19.828099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63f8cf-7340-4e69-9c2f-a877053fa3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:19.875667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-da63f8cf-7340-4e69-9c2f-a877053fa3ce None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:19.876239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-da63f8cf-7340-4e69-9c2f-a877053fa3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:19.885648 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1117/2221] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:19.894235 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b6651-b15e-49d4-b290-2efb73a8cfcc None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:19.894613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b6651-b15e-49d4-b290-2efb73a8cfcc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:19.894968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b6651-b15e-49d4-b290-2efb73a8cfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:19.918744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ea0c2-27cb-4a1e-8d70-09390cc5d3f3 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:19.924330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:19.925285 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:19.925576 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:19.927793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] PATH_INFO: `/v3/projects/062fa2d348eb41b59501286b11233bca` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:19.930217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=062fa2d348eb41b59501286b11233bca)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:19.932282 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:19.945239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-ab7b6651-b15e-49d4-b290-2efb73a8cfcc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:19.950720 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ab7b6651-b15e-49d4-b290-2efb73a8cfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:19.973044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1118/2222] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:48:19.973044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b6651-b15e-49d4-b290-2efb73a8cfcc None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:19.973044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ab7b6651-b15e-49d4-b290-2efb73a8cfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:20.034771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '062fa2d348eb41b59501286b11233bca', 'request_id': 'req-c8fba925-1f31-4834-8b24-a9b919e5be1c'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.050678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '062fa2d348eb41b59501286b11233bca', 'request_id': 'req-c8fba925-1f31-4834-8b24-a9b919e5be1c'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.057568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '062fa2d348eb41b59501286b11233bca', 'request_id': 'req-c8fba925-1f31-4834-8b24-a9b919e5be1c'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.069799 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] The token cache is being invalidate because project 062fa2d348eb41b59501286b11233bca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:20.070096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.070796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.076982 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1105/2223] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:19 2022] DELETE /identity/v3/projects/062fa2d348eb41b59501286b11233bca => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:20.077945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:20.079602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:20.110669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b6651-b15e-49d4-b290-2efb73a8cfcc None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:20.181121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fba925-1f31-4834-8b24-a9b919e5be1c admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:20.186754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c8782-92a4-4c44-92d4-191d80b4a0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:20.187424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c8782-92a4-4c44-92d4-191d80b4a0ba admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:20.187708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c8782-92a4-4c44-92d4-191d80b4a0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:20.187983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c8782-92a4-4c44-92d4-191d80b4a0ba admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:20.188741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c8782-92a4-4c44-92d4-191d80b4a0ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:48:20.189161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c8782-92a4-4c44-92d4-191d80b4a0ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:20.190786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c8782-92a4-4c44-92d4-191d80b4a0ba admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:20.204621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1106/2224] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:48:20.205459 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c8782-92a4-4c44-92d4-191d80b4a0ba admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:20.207163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f19c8782-92a4-4c44-92d4-191d80b4a0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:20.208470 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d50bf-ebbc-425e-9caa-5392c913d2af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:20.209589 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d50bf-ebbc-425e-9caa-5392c913d2af service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:20.210178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d50bf-ebbc-425e-9caa-5392c913d2af service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:20.211313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d50bf-ebbc-425e-9caa-5392c913d2af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:20.214559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-373d50bf-ebbc-425e-9caa-5392c913d2af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:20.222525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c8782-92a4-4c44-92d4-191d80b4a0ba admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:20.228755 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:20.229715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:20.230354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:20.230354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] PATH_INFO: `/v3/users/9d717fc739eb4a1fb84e53a06b5e9ff6` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:20.231700 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d717fc739eb4a1fb84e53a06b5e9ff6)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:20.234812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:20.240343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d50bf-ebbc-425e-9caa-5392c913d2af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:20.242974 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d50bf-ebbc-425e-9caa-5392c913d2af service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:20.256150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1119/2225] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:20.257306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d50bf-ebbc-425e-9caa-5392c913d2af service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:20.272089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d50bf-ebbc-425e-9caa-5392c913d2af service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:20.276804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:20.281062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac89c98-fc70-4b0e-92a9-a5372d2a0629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:20.282146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac89c98-fc70-4b0e-92a9-a5372d2a0629 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:20.282400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac89c98-fc70-4b0e-92a9-a5372d2a0629 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:20.284422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac89c98-fc70-4b0e-92a9-a5372d2a0629 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:20.285402 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac89c98-fc70-4b0e-92a9-a5372d2a0629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:20.287710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac89c98-fc70-4b0e-92a9-a5372d2a0629 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:20.323401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d717fc739eb4a1fb84e53a06b5e9ff6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.326255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1120/2226] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:20 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:20.330255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac89c98-fc70-4b0e-92a9-a5372d2a0629 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:20.335513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d717fc739eb4a1fb84e53a06b5e9ff6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.346567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:20.354390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d717fc739eb4a1fb84e53a06b5e9ff6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.365675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d717fc739eb4a1fb84e53a06b5e9ff6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.367482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d717fc739eb4a1fb84e53a06b5e9ff6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.380557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1107/2227] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:20 2022] DELETE /identity/v3/users/9d717fc739eb4a1fb84e53a06b5e9ff6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:20.457908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac89c98-fc70-4b0e-92a9-a5372d2a0629 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:20.575086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a4159-9238-4a0e-a0c4-c063fc92f14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:20.576002 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a4159-9238-4a0e-a0c4-c063fc92f14d admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:20.577406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a4159-9238-4a0e-a0c4-c063fc92f14d admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:20.577795 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a4159-9238-4a0e-a0c4-c063fc92f14d admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:20.579251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a4159-9238-4a0e-a0c4-c063fc92f14d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:20.581438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a4159-9238-4a0e-a0c4-c063fc92f14d admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:20.617378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1121/2228] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:20 2022] POST /identity/v3/users => generated 294 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:48:20.621442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:20.624041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:20.653159 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ba142-c08e-4337-a4a3-2a7e01fdb6cc admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:20.660421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013dbbee-e9e1-46b8-917c-c93a7bfe4ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:20.661269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dbbee-e9e1-46b8-917c-c93a7bfe4ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:20.661654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dbbee-e9e1-46b8-917c-c93a7bfe4ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:20.662033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dbbee-e9e1-46b8-917c-c93a7bfe4ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:20.662956 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013dbbee-e9e1-46b8-917c-c93a7bfe4ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:20.667697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013dbbee-e9e1-46b8-917c-c93a7bfe4ee7 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:20.688535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1108/2229] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:20.690147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a4159-9238-4a0e-a0c4-c063fc92f14d admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:20.710132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a4159-9238-4a0e-a0c4-c063fc92f14d admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:20.721598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec7e40-4099-4322-8495-c6b851ec0310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:20.724479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec7e40-4099-4322-8495-c6b851ec0310 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:20.725824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec7e40-4099-4322-8495-c6b851ec0310 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:20.726442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec7e40-4099-4322-8495-c6b851ec0310 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:20.728756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec7e40-4099-4322-8495-c6b851ec0310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:20.731459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec7e40-4099-4322-8495-c6b851ec0310 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:20.774140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013dbbee-e9e1-46b8-917c-c93a7bfe4ee7 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:20.776408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1122/2230] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:20.790840 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013dbbee-e9e1-46b8-917c-c93a7bfe4ee7 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:20.801585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552917a0-9934-4156-9f69-d5d16452cc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:20.802440 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552917a0-9934-4156-9f69-d5d16452cc88 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:20.802662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552917a0-9934-4156-9f69-d5d16452cc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:20.802870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552917a0-9934-4156-9f69-d5d16452cc88 admin admin] PATH_INFO: `/v3/projects/93630af4f68a492482f2aaec4783d1aa/users/78b5ac3846474b12a8eb7257a5307083/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:20.823607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec7e40-4099-4322-8495-c6b851ec0310 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:20.827355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c1ec7e40-4099-4322-8495-c6b851ec0310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:20.843045 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552917a0-9934-4156-9f69-d5d16452cc88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93630af4f68a492482f2aaec4783d1aa, user_id=78b5ac3846474b12a8eb7257a5307083, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:20.848280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552917a0-9934-4156-9f69-d5d16452cc88 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:20.861214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec7e40-4099-4322-8495-c6b851ec0310 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:20.868388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:20.870120 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:20.870409 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:20.870824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] PATH_INFO: `/v3/projects/a36b528e07d148b3bc9c24a96b34c738` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:20.872295 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a36b528e07d148b3bc9c24a96b34c738)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:20.874770 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1109/2231] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:20 2022] PUT /identity/v3/projects/93630af4f68a492482f2aaec4783d1aa/users/78b5ac3846474b12a8eb7257a5307083/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:48:20.883928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:20.885770 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf2da0-3330-427d-ba45-1a99cf279c68 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:20.886071 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf2da0-3330-427d-ba45-1a99cf279c68 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:20.886356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf2da0-3330-427d-ba45-1a99cf279c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:20.940877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-56bf2da0-3330-427d-ba45-1a99cf279c68 None None] MFA Rules not processed for user `78b5ac3846474b12a8eb7257a5307083`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:20.942856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-56bf2da0-3330-427d-ba45-1a99cf279c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:20.950879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a36b528e07d148b3bc9c24a96b34c738', 'request_id': 'req-4789f876-272c-46e3-97bf-3aa83fc4247b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.961374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a36b528e07d148b3bc9c24a96b34c738', 'request_id': 'req-4789f876-272c-46e3-97bf-3aa83fc4247b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.969641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a36b528e07d148b3bc9c24a96b34c738', 'request_id': 'req-4789f876-272c-46e3-97bf-3aa83fc4247b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:20.999670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] The token cache is being invalidate because project a36b528e07d148b3bc9c24a96b34c738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:20.999965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:21.000709 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:21.008377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1123/2232] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:20 2022] DELETE /identity/v3/projects/a36b528e07d148b3bc9c24a96b34c738 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:21.009340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:21.010911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:21.052523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1110/2233] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:21.056853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf2da0-3330-427d-ba45-1a99cf279c68 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:21.058395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-56bf2da0-3330-427d-ba45-1a99cf279c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:21.061825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:21.147614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789f876-272c-46e3-97bf-3aa83fc4247b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:21.159940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:21.161153 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:21.161454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:21.162339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] PATH_INFO: `/v3/users/1a1392f0898f4afa966976f912a9b74b` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:21.165565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a1392f0898f4afa966976f912a9b74b)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:21.167454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:21.193877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:21.203826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf2da0-3330-427d-ba45-1a99cf279c68 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:21.220408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb660d0-2af4-4dba-a871-b516224247a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:21.221235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb660d0-2af4-4dba-a871-b516224247a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:21.221525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb660d0-2af4-4dba-a871-b516224247a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:21.221803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb660d0-2af4-4dba-a871-b516224247a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:21.222567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb660d0-2af4-4dba-a871-b516224247a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:21.224771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb660d0-2af4-4dba-a871-b516224247a3 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:21.281937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a1392f0898f4afa966976f912a9b74b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:21.282750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a1392f0898f4afa966976f912a9b74b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:21.308045 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a1392f0898f4afa966976f912a9b74b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:21.315176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1111/2234] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:21 2022] POST /identity/v3/projects => generated 387 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:21.318001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb660d0-2af4-4dba-a871-b516224247a3 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:21.332871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a1392f0898f4afa966976f912a9b74b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:21.351907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a1392f0898f4afa966976f912a9b74b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:21.367604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1124/2235] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:21 2022] DELETE /identity/v3/users/1a1392f0898f4afa966976f912a9b74b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:21.487880 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb660d0-2af4-4dba-a871-b516224247a3 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:21.641943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2ee35-395f-48e7-b2fb-96865c9b185b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:21.644190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2ee35-395f-48e7-b2fb-96865c9b185b admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:21.644190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2ee35-395f-48e7-b2fb-96865c9b185b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:21.644190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2ee35-395f-48e7-b2fb-96865c9b185b admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:21.644190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a2ee35-395f-48e7-b2fb-96865c9b185b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:21.651553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a2ee35-395f-48e7-b2fb-96865c9b185b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:21.693794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1112/2236] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:21 2022] POST /identity/v3/users => generated 300 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:21.698073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:21.699603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:21.736697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe91bf4-5387-4171-a831-c2db6ad423eb admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:21.742764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8532d4-73e5-47bc-b842-f544350d7e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:21.743459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8532d4-73e5-47bc-b842-f544350d7e97 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:21.743744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8532d4-73e5-47bc-b842-f544350d7e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:21.744031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8532d4-73e5-47bc-b842-f544350d7e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:21.745116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8532d4-73e5-47bc-b842-f544350d7e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:21.746977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8532d4-73e5-47bc-b842-f544350d7e97 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:21.763297 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1125/2237] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:21.767858 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2ee35-395f-48e7-b2fb-96865c9b185b admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:21.780276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2ee35-395f-48e7-b2fb-96865c9b185b admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:21.785470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43af74-30f0-420f-95ea-dc39a5498940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:21.786517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43af74-30f0-420f-95ea-dc39a5498940 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:21.786822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43af74-30f0-420f-95ea-dc39a5498940 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:21.787621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43af74-30f0-420f-95ea-dc39a5498940 admin admin] PATH_INFO: `/v3/projects/9cab25aaa28146fd8c15a4b3bb4c4619/users/6ce686b46de44b609bd03a580afefab3/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:21.806248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43af74-30f0-420f-95ea-dc39a5498940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cab25aaa28146fd8c15a4b3bb4c4619, user_id=6ce686b46de44b609bd03a580afefab3, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:21.808998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43af74-30f0-420f-95ea-dc39a5498940 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:21.838664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1113/2238] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:21 2022] PUT /identity/v3/projects/9cab25aaa28146fd8c15a4b3bb4c4619/users/6ce686b46de44b609bd03a580afefab3/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:21.843140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8532d4-73e5-47bc-b842-f544350d7e97 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:21.857247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8532d4-73e5-47bc-b842-f544350d7e97 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:21.953043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadeea0-74ba-4c04-b1c1-ae2ee45aab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:21.953785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadeea0-74ba-4c04-b1c1-ae2ee45aab3e admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:21.954071 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadeea0-74ba-4c04-b1c1-ae2ee45aab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:21.954348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadeea0-74ba-4c04-b1c1-ae2ee45aab3e admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:21.955350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadeea0-74ba-4c04-b1c1-ae2ee45aab3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:21.957331 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadeea0-74ba-4c04-b1c1-ae2ee45aab3e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:21.977705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43af74-30f0-420f-95ea-dc39a5498940 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:21.978439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1126/2239] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:21.993988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43af74-30f0-420f-95ea-dc39a5498940 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:22.002542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d299f5-358c-47a1-a535-0d6d5e134b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:22.003650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d299f5-358c-47a1-a535-0d6d5e134b55 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:22.003785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d299f5-358c-47a1-a535-0d6d5e134b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:22.004000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d299f5-358c-47a1-a535-0d6d5e134b55 admin admin] PATH_INFO: `/v3/projects/9cab25aaa28146fd8c15a4b3bb4c4619/users/6ce686b46de44b609bd03a580afefab3/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:22.007127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d299f5-358c-47a1-a535-0d6d5e134b55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cab25aaa28146fd8c15a4b3bb4c4619, user_id=6ce686b46de44b609bd03a580afefab3, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:22.011101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d299f5-358c-47a1-a535-0d6d5e134b55 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:22.046256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1114/2240] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:21 2022] PUT /identity/v3/projects/9cab25aaa28146fd8c15a4b3bb4c4619/users/6ce686b46de44b609bd03a580afefab3/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:48:22.047473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6c9e5-5373-4967-8c19-21757abee9a6 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:22.048137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6c9e5-5373-4967-8c19-21757abee9a6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:22.048137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6c9e5-5373-4967-8c19-21757abee9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:22.081009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d5e6c9e5-5373-4967-8c19-21757abee9a6 None None] MFA Rules not processed for user `6ce686b46de44b609bd03a580afefab3`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:22.083500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d5e6c9e5-5373-4967-8c19-21757abee9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:22.098152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:22.177076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d299f5-358c-47a1-a535-0d6d5e134b55 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:22.181648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a4d299f5-358c-47a1-a535-0d6d5e134b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:22.264535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1127/2241] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:22.279461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6c9e5-5373-4967-8c19-21757abee9a6 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:22.281016 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d5e6c9e5-5373-4967-8c19-21757abee9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:22.327537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:22.350688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6c9e5-5373-4967-8c19-21757abee9a6 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:22.355033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b751f-507e-411a-9b00-f3dd0e770db3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:22.355699 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b751f-507e-411a-9b00-f3dd0e770db3 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:22.355987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b751f-507e-411a-9b00-f3dd0e770db3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:22.356378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b751f-507e-411a-9b00-f3dd0e770db3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:22.362128 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d299f5-358c-47a1-a535-0d6d5e134b55 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:22.378804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:22.380126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:22.380445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:22.381251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] PATH_INFO: `/v3/projects/d1b7f615f45244c489400889ce931735` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:22.383221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1b7f615f45244c489400889ce931735)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:22.388381 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b751f-507e-411a-9b00-f3dd0e770db3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:22.391425 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b751f-507e-411a-9b00-f3dd0e770db3 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:22.397632 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:22.424416 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1128/2242] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:48:22.548683 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1b7f615f45244c489400889ce931735', 'request_id': 'req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:22.572136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1b7f615f45244c489400889ce931735', 'request_id': 'req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:22.587567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1b7f615f45244c489400889ce931735', 'request_id': 'req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:22.611083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] The token cache is being invalidate because project d1b7f615f45244c489400889ce931735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:22.611383 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:22.612464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:22.617458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1115/2243] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:22 2022] DELETE /identity/v3/projects/d1b7f615f45244c489400889ce931735 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:22.624555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d53cb-eb4a-4161-a8be-b0b15665a197 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:22.625302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d53cb-eb4a-4161-a8be-b0b15665a197 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:22.625623 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d53cb-eb4a-4161-a8be-b0b15665a197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:22.661751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-570d53cb-eb4a-4161-a8be-b0b15665a197 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:22.667142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-570d53cb-eb4a-4161-a8be-b0b15665a197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:22.672855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:22.674304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:22.870005 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1129/2244] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:22.873863 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1afd74-f2dc-4d8c-b96f-36b1b2246e12 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:22.879723 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e34dd-cbf6-47c5-b32d-c2c3d10e27dc None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:22.880060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36525193-64ac-40f4-9b1d-db7bf93dcd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:22.880217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e34dd-cbf6-47c5-b32d-c2c3d10e27dc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:22.880510 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e34dd-cbf6-47c5-b32d-c2c3d10e27dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:22.881488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36525193-64ac-40f4-9b1d-db7bf93dcd78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:22.881988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36525193-64ac-40f4-9b1d-db7bf93dcd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:22.882474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36525193-64ac-40f4-9b1d-db7bf93dcd78 admin admin] PATH_INFO: `/v3/users/78b5ac3846474b12a8eb7257a5307083` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:22.886950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36525193-64ac-40f4-9b1d-db7bf93dcd78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b5ac3846474b12a8eb7257a5307083)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:22.890323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36525193-64ac-40f4-9b1d-db7bf93dcd78 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:22.906942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-8f6e34dd-cbf6-47c5-b32d-c2c3d10e27dc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:22.908836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8f6e34dd-cbf6-47c5-b32d-c2c3d10e27dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:22.918399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1130/2245] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:22.922809 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e34dd-cbf6-47c5-b32d-c2c3d10e27dc None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:22.942778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e34dd-cbf6-47c5-b32d-c2c3d10e27dc None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:22.948604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed210f-2a2c-461e-baf2-59750aa655f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:22.949904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed210f-2a2c-461e-baf2-59750aa655f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:22.950200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed210f-2a2c-461e-baf2-59750aa655f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:22.950484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed210f-2a2c-461e-baf2-59750aa655f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:22.951299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed210f-2a2c-461e-baf2-59750aa655f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:48:22.951636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed210f-2a2c-461e-baf2-59750aa655f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:22.953319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed210f-2a2c-461e-baf2-59750aa655f3 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:22.966420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-36525193-64ac-40f4-9b1d-db7bf93dcd78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b5ac3846474b12a8eb7257a5307083'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:22.973704 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1131/2246] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:48:22.978201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed210f-2a2c-461e-baf2-59750aa655f3 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:22.985992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-36525193-64ac-40f4-9b1d-db7bf93dcd78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b5ac3846474b12a8eb7257a5307083'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:22.991070 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed210f-2a2c-461e-baf2-59750aa655f3 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:23.004802 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30105418-c211-461a-9c67-5bbf74b6889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:23.005417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30105418-c211-461a-9c67-5bbf74b6889b admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:23.005703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30105418-c211-461a-9c67-5bbf74b6889b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:23.006036 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30105418-c211-461a-9c67-5bbf74b6889b admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:23.006751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30105418-c211-461a-9c67-5bbf74b6889b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:23.009089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-36525193-64ac-40f4-9b1d-db7bf93dcd78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b5ac3846474b12a8eb7257a5307083'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:23.012557 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30105418-c211-461a-9c67-5bbf74b6889b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:23.017600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:23.021005 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-36525193-64ac-40f4-9b1d-db7bf93dcd78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b5ac3846474b12a8eb7257a5307083'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:23.022303 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-36525193-64ac-40f4-9b1d-db7bf93dcd78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b5ac3846474b12a8eb7257a5307083'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:23.031885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1116/2247] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:22 2022] DELETE /identity/v3/users/78b5ac3846474b12a8eb7257a5307083 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:23.037706 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5accdeb-1239-4d71-bbfb-bf3ae20ded2b None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:23.038217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5accdeb-1239-4d71-bbfb-bf3ae20ded2b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:23.038601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5accdeb-1239-4d71-bbfb-bf3ae20ded2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:23.073753 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-b5accdeb-1239-4d71-bbfb-bf3ae20ded2b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:23.076096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b5accdeb-1239-4d71-bbfb-bf3ae20ded2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:23.093428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1132/2248] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:22 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:23.094491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30105418-c211-461a-9c67-5bbf74b6889b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:23.101780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-30105418-c211-461a-9c67-5bbf74b6889b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:23.129617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:23.223915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30105418-c211-461a-9c67-5bbf74b6889b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:23.231987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2055ca-541e-41b3-903c-8ca7600367b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:23.233707 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2055ca-541e-41b3-903c-8ca7600367b3 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:23.234291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2055ca-541e-41b3-903c-8ca7600367b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:23.235124 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2055ca-541e-41b3-903c-8ca7600367b3 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:23.236247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2055ca-541e-41b3-903c-8ca7600367b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:23.238897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2055ca-541e-41b3-903c-8ca7600367b3 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:23.259031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1117/2249] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:23.287645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1133/2250] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:23 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:23.291850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5accdeb-1239-4d71-bbfb-bf3ae20ded2b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:23.309936 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5accdeb-1239-4d71-bbfb-bf3ae20ded2b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:23.319788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877b2a5-9044-4f34-8f75-081cbc518d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:23.320492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877b2a5-9044-4f34-8f75-081cbc518d14 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:23.320832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877b2a5-9044-4f34-8f75-081cbc518d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:23.321108 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877b2a5-9044-4f34-8f75-081cbc518d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:23.328122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877b2a5-9044-4f34-8f75-081cbc518d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:23.330129 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877b2a5-9044-4f34-8f75-081cbc518d14 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:23.332243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00e061-f3b0-4c44-9ff9-7238efddd17f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:23.332563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00e061-f3b0-4c44-9ff9-7238efddd17f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:23.333027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00e061-f3b0-4c44-9ff9-7238efddd17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:23.381194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1118/2251] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:23.382155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877b2a5-9044-4f34-8f75-081cbc518d14 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:23.385818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3877b2a5-9044-4f34-8f75-081cbc518d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:23.406875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7d00e061-f3b0-4c44-9ff9-7238efddd17f None None] MFA Rules not processed for user `d3f0b30685e94f83883d682731644a89`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:23.408139 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7d00e061-f3b0-4c44-9ff9-7238efddd17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:23.505983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877b2a5-9044-4f34-8f75-081cbc518d14 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:23.510696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0ada2-38bc-403c-b96e-b8f8b5752700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:23.511483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0ada2-38bc-403c-b96e-b8f8b5752700 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:23.512278 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0ada2-38bc-403c-b96e-b8f8b5752700 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:23.512278 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0ada2-38bc-403c-b96e-b8f8b5752700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:23.527848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0ada2-38bc-403c-b96e-b8f8b5752700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:23.530289 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0ada2-38bc-403c-b96e-b8f8b5752700 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:23.545009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1119/2252] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:23.545797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0ada2-38bc-403c-b96e-b8f8b5752700 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:23.558543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0ada2-38bc-403c-b96e-b8f8b5752700 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:23.564393 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90f733-3290-42fd-bd0b-763ed4c55326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:23.565851 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90f733-3290-42fd-bd0b-763ed4c55326 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:23.566139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90f733-3290-42fd-bd0b-763ed4c55326 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:23.566410 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90f733-3290-42fd-bd0b-763ed4c55326 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:23.567338 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90f733-3290-42fd-bd0b-763ed4c55326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:23.569355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90f733-3290-42fd-bd0b-763ed4c55326 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:23.570828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1134/2253] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:48:23.579565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00e061-f3b0-4c44-9ff9-7238efddd17f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:23.581305 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7d00e061-f3b0-4c44-9ff9-7238efddd17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:23.582976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1120/2254] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:23.588095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90f733-3290-42fd-bd0b-763ed4c55326 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:23.616439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90f733-3290-42fd-bd0b-763ed4c55326 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:23.632969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40bf1a-04f6-4d8e-97c3-ea0214c19204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:23.633899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40bf1a-04f6-4d8e-97c3-ea0214c19204 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:23.634185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40bf1a-04f6-4d8e-97c3-ea0214c19204 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:23.636882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40bf1a-04f6-4d8e-97c3-ea0214c19204 admin admin] PATH_INFO: `/v3/projects/1e76bc803d784442936b25a7ad36d564/users/316c65a9dcd8472ab10b532537e5dad7/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:23.657422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40bf1a-04f6-4d8e-97c3-ea0214c19204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e76bc803d784442936b25a7ad36d564, user_id=316c65a9dcd8472ab10b532537e5dad7, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:23.664074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40bf1a-04f6-4d8e-97c3-ea0214c19204 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:23.705422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1121/2255] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:23 2022] PUT /identity/v3/projects/1e76bc803d784442936b25a7ad36d564/users/316c65a9dcd8472ab10b532537e5dad7/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:23.711096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358c8ef-27f2-4bab-bc1f-7e811246240a None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:23.711530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358c8ef-27f2-4bab-bc1f-7e811246240a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:23.712854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358c8ef-27f2-4bab-bc1f-7e811246240a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:23.743647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-0358c8ef-27f2-4bab-bc1f-7e811246240a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:23.748516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0358c8ef-27f2-4bab-bc1f-7e811246240a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:23.862259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:23.892029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1122/2256] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:23.902261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358c8ef-27f2-4bab-bc1f-7e811246240a None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:23.907254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0358c8ef-27f2-4bab-bc1f-7e811246240a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:23.923278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00e061-f3b0-4c44-9ff9-7238efddd17f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:23.962761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:24.044612 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae4c53-d2cf-40cd-92ba-9fefa3a5e260 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:24.051700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae4c53-d2cf-40cd-92ba-9fefa3a5e260 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:24.051700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae4c53-d2cf-40cd-92ba-9fefa3a5e260 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:24.051700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae4c53-d2cf-40cd-92ba-9fefa3a5e260 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:24.110100 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358c8ef-27f2-4bab-bc1f-7e811246240a None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:24.120608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b93e7-35e2-41bc-ac26-446e356dd3e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:24.121965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b93e7-35e2-41bc-ac26-446e356dd3e8 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:24.124878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b93e7-35e2-41bc-ac26-446e356dd3e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:24.128667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b93e7-35e2-41bc-ac26-446e356dd3e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:24.149030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b93e7-35e2-41bc-ac26-446e356dd3e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:24.150778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b93e7-35e2-41bc-ac26-446e356dd3e8 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:24.168193 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1123/2257] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:48:24.208609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae4c53-d2cf-40cd-92ba-9fefa3a5e260 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:24.210845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae4c53-d2cf-40cd-92ba-9fefa3a5e260 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:24.237842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1135/2258] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:24.869429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b93e7-35e2-41bc-ac26-446e356dd3e8 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:24.871926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-776b93e7-35e2-41bc-ac26-446e356dd3e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:24.909257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b93e7-35e2-41bc-ac26-446e356dd3e8 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:24.918245 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:24.919666 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:24.920005 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:24.923930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] PATH_INFO: `/v3/projects/93630af4f68a492482f2aaec4783d1aa` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:24.927817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93630af4f68a492482f2aaec4783d1aa)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:24.930168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:25.048011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93630af4f68a492482f2aaec4783d1aa', 'request_id': 'req-11face4c-78e2-4963-a750-3388eddb49c6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:25.071734 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93630af4f68a492482f2aaec4783d1aa', 'request_id': 'req-11face4c-78e2-4963-a750-3388eddb49c6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:25.082528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93630af4f68a492482f2aaec4783d1aa', 'request_id': 'req-11face4c-78e2-4963-a750-3388eddb49c6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:25.095635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] The token cache is being invalidate because project 93630af4f68a492482f2aaec4783d1aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:25.095929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:25.096694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:25.098578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1124/2259] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:24 2022] DELETE /identity/v3/projects/93630af4f68a492482f2aaec4783d1aa => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:25.102343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae4c53-d2cf-40cd-92ba-9fefa3a5e260 service nova] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:25.116332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae4c53-d2cf-40cd-92ba-9fefa3a5e260 service nova] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:25.255154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3883eae-8b0b-4018-9602-9229654452e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:25.256885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3883eae-8b0b-4018-9602-9229654452e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:25.257277 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3883eae-8b0b-4018-9602-9229654452e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:25.257905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3883eae-8b0b-4018-9602-9229654452e7 admin admin] PATH_INFO: `/v3/users/6ce686b46de44b609bd03a580afefab3` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:25.260711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3883eae-8b0b-4018-9602-9229654452e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ce686b46de44b609bd03a580afefab3)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:25.264314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3883eae-8b0b-4018-9602-9229654452e7 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:25.364265 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e3883eae-8b0b-4018-9602-9229654452e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ce686b46de44b609bd03a580afefab3'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:25.366036 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e3883eae-8b0b-4018-9602-9229654452e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ce686b46de44b609bd03a580afefab3'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:25.388481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e3883eae-8b0b-4018-9602-9229654452e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ce686b46de44b609bd03a580afefab3'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:25.403394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:25.409248 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e3883eae-8b0b-4018-9602-9229654452e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ce686b46de44b609bd03a580afefab3'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:25.426587 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e3883eae-8b0b-4018-9602-9229654452e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ce686b46de44b609bd03a580afefab3'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:25.442415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1136/2260] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:25 2022] DELETE /identity/v3/users/6ce686b46de44b609bd03a580afefab3 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:26.504532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:26.507651 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:26.524302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11face4c-78e2-4963-a750-3388eddb49c6 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:26.705229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:26.706127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:26.706406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:26.706857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028 admin admin] PATH_INFO: `/v3/projects/9cab25aaa28146fd8c15a4b3bb4c4619` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:26.710749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cab25aaa28146fd8c15a4b3bb4c4619)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:26.713152 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:26.757201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:26.850483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cab25aaa28146fd8c15a4b3bb4c4619', 'request_id': 'req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:26.863007 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cab25aaa28146fd8c15a4b3bb4c4619', 'request_id': 'req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:26.875713 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cab25aaa28146fd8c15a4b3bb4c4619', 'request_id': 'req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:26.903074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028 admin admin] The token cache is being invalidate because project 9cab25aaa28146fd8c15a4b3bb4c4619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:26.903370 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:26.907230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9b5f0ab9-ebdd-41a4-96d5-726e6aef2028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:26.912155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1125/2261] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:26 2022] DELETE /identity/v3/projects/9cab25aaa28146fd8c15a4b3bb4c4619 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:26.917020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b26d74-0088-4923-8c32-f44bb1225c5b None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:26.918015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b26d74-0088-4923-8c32-f44bb1225c5b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:26.918522 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b26d74-0088-4923-8c32-f44bb1225c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:26.951039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-97b26d74-0088-4923-8c32-f44bb1225c5b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:26.953480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-97b26d74-0088-4923-8c32-f44bb1225c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:26.968372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e211c0b-996a-4e25-be17-42c8aa09cb8c None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:26.968718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e211c0b-996a-4e25-be17-42c8aa09cb8c None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:26.968997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e211c0b-996a-4e25-be17-42c8aa09cb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:27.027650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-0e211c0b-996a-4e25-be17-42c8aa09cb8c None None] MFA Rules not processed for user `316c65a9dcd8472ab10b532537e5dad7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:27.029704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0e211c0b-996a-4e25-be17-42c8aa09cb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:27.074083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1137/2262] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:27.075795 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15fdde-a7a8-4626-8990-114418e8becb None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:27.076080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15fdde-a7a8-4626-8990-114418e8becb None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:27.076360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15fdde-a7a8-4626-8990-114418e8becb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:27.101519 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7f15fdde-a7a8-4626-8990-114418e8becb None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:27.103382 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7f15fdde-a7a8-4626-8990-114418e8becb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:27.121091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1138/2263] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:27.122183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15fdde-a7a8-4626-8990-114418e8becb None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:27.141414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15fdde-a7a8-4626-8990-114418e8becb None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:27.156611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f443a-58ed-436a-93ee-849b6369f292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:27.160238 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f443a-58ed-436a-93ee-849b6369f292 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:27.160557 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f443a-58ed-436a-93ee-849b6369f292 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:27.160869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f443a-58ed-436a-93ee-849b6369f292 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:27.163223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f443a-58ed-436a-93ee-849b6369f292 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:48:27.164012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f443a-58ed-436a-93ee-849b6369f292 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:27.168105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f443a-58ed-436a-93ee-849b6369f292 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:27.196775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1126/2264] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:48:27.208598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e211c0b-996a-4e25-be17-42c8aa09cb8c None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:27.209871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1139/2265] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:48:27.210378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0e211c0b-996a-4e25-be17-42c8aa09cb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:27.215671 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f443a-58ed-436a-93ee-849b6369f292 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:27.238796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f443a-58ed-436a-93ee-849b6369f292 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:27.249875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b8954-8913-4f29-9cba-0ffc0e0dc71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:27.250638 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b8954-8913-4f29-9cba-0ffc0e0dc71a admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:27.250930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b8954-8913-4f29-9cba-0ffc0e0dc71a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:27.251208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b8954-8913-4f29-9cba-0ffc0e0dc71a admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:27.253159 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b8954-8913-4f29-9cba-0ffc0e0dc71a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:27.257768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b8954-8913-4f29-9cba-0ffc0e0dc71a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:27.332085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1140/2266] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:27 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:27.334595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a3880-8511-4765-aebc-fdfd2513bfb3 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:27.336068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a3880-8511-4765-aebc-fdfd2513bfb3 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:27.336313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a3880-8511-4765-aebc-fdfd2513bfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:27.361542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e211c0b-996a-4e25-be17-42c8aa09cb8c None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:27.380361 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-212a3880-8511-4765-aebc-fdfd2513bfb3 None None] MFA Rules not processed for user `d3f0b30685e94f83883d682731644a89`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:27.394624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-212a3880-8511-4765-aebc-fdfd2513bfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:27.466150 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056e4dc-8abb-4fa1-9855-99a944ead18e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:27.469591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056e4dc-8abb-4fa1-9855-99a944ead18e service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:27.470176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056e4dc-8abb-4fa1-9855-99a944ead18e service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:27.471877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056e4dc-8abb-4fa1-9855-99a944ead18e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:27.484686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:27.496654 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1141/2267] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:27.496654 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a3880-8511-4765-aebc-fdfd2513bfb3 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:27.630460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056e4dc-8abb-4fa1-9855-99a944ead18e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:27.632887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056e4dc-8abb-4fa1-9855-99a944ead18e service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:27.644599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1127/2268] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:48:27.651735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:27.673659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a3880-8511-4765-aebc-fdfd2513bfb3 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:27.680868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f2b6b-5986-427c-ada8-2d73bf556e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:27.681870 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f2b6b-5986-427c-ada8-2d73bf556e5b admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:27.682162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f2b6b-5986-427c-ada8-2d73bf556e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:27.682442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f2b6b-5986-427c-ada8-2d73bf556e5b admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:27.683152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f2b6b-5986-427c-ada8-2d73bf556e5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:27.686829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f2b6b-5986-427c-ada8-2d73bf556e5b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:27.729482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1142/2269] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:27 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:27.733554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056e4dc-8abb-4fa1-9855-99a944ead18e service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:27.745598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056e4dc-8abb-4fa1-9855-99a944ead18e service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:27.760498 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466ac88-c1de-4736-ad40-8b78d5400e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:27.761495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ac88-c1de-4736-ad40-8b78d5400e55 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:27.761930 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ac88-c1de-4736-ad40-8b78d5400e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:27.762391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ac88-c1de-4736-ad40-8b78d5400e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:27.763284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466ac88-c1de-4736-ad40-8b78d5400e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:27.765929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466ac88-c1de-4736-ad40-8b78d5400e55 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:27.790682 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1128/2270] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:27.795053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f2b6b-5986-427c-ada8-2d73bf556e5b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:27.820543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f2b6b-5986-427c-ada8-2d73bf556e5b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:27.836270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9043fd-fc45-4242-aefa-b0518d30260e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:27.838812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9043fd-fc45-4242-aefa-b0518d30260e admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:27.839219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9043fd-fc45-4242-aefa-b0518d30260e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:27.840044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9043fd-fc45-4242-aefa-b0518d30260e admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:27.844119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9043fd-fc45-4242-aefa-b0518d30260e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:27.849295 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9043fd-fc45-4242-aefa-b0518d30260e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:27.889045 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1143/2271] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:27.892854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466ac88-c1de-4736-ad40-8b78d5400e55 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:27.907655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466ac88-c1de-4736-ad40-8b78d5400e55 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:27.920397 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3238cdf-c4db-4c55-8ca7-34064877931c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:27.922413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3238cdf-c4db-4c55-8ca7-34064877931c admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:27.924268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3238cdf-c4db-4c55-8ca7-34064877931c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:27.925525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3238cdf-c4db-4c55-8ca7-34064877931c admin admin] PATH_INFO: `/v3/projects/c5ee6a6d38bd4801aba2f913fc51838e/users/851d590710c049ea842796e8869c1c97/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:27.968206 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3238cdf-c4db-4c55-8ca7-34064877931c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ee6a6d38bd4801aba2f913fc51838e, user_id=851d590710c049ea842796e8869c1c97, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:27.970951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3238cdf-c4db-4c55-8ca7-34064877931c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:28.003526 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1129/2272] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:27 2022] PUT /identity/v3/projects/c5ee6a6d38bd4801aba2f913fc51838e/users/851d590710c049ea842796e8869c1c97/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:48:28.015576 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942f422-de8f-4790-99f3-f63e8ac8c9d8 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:28.016104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942f422-de8f-4790-99f3-f63e8ac8c9d8 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:28.016198 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942f422-de8f-4790-99f3-f63e8ac8c9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:28.077581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-6942f422-de8f-4790-99f3-f63e8ac8c9d8 None None] MFA Rules not processed for user `851d590710c049ea842796e8869c1c97`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:28.081289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6942f422-de8f-4790-99f3-f63e8ac8c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:28.297763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1144/2273] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:28.315906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3238cdf-c4db-4c55-8ca7-34064877931c admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:28.410385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3238cdf-c4db-4c55-8ca7-34064877931c admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:28.419258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75141e-035a-4355-83df-b0585a83afac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:28.420491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75141e-035a-4355-83df-b0585a83afac service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:28.420803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75141e-035a-4355-83df-b0585a83afac service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:28.421451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75141e-035a-4355-83df-b0585a83afac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:28.441626 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75141e-035a-4355-83df-b0585a83afac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:28.443089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75141e-035a-4355-83df-b0585a83afac service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:28.463720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1130/2274] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:28.501331 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942f422-de8f-4790-99f3-f63e8ac8c9d8 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:28.603455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942f422-de8f-4790-99f3-f63e8ac8c9d8 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:28.609708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:28.610950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:28.611894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:28.611894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] PATH_INFO: `/v3/users/851d590710c049ea842796e8869c1c97` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:28.613295 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=851d590710c049ea842796e8869c1c97)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:28.615533 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:28.697684 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '851d590710c049ea842796e8869c1c97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:28.698416 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '851d590710c049ea842796e8869c1c97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:28.714896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '851d590710c049ea842796e8869c1c97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:28.729221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:28.745650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '851d590710c049ea842796e8869c1c97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:28.785409 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '851d590710c049ea842796e8869c1c97'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:28.794052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1145/2275] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:28 2022] DELETE /identity/v3/users/851d590710c049ea842796e8869c1c97 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:28.798906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23847759-8ba5-4219-a0c2-d3d0f03eee22 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:28.799761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23847759-8ba5-4219-a0c2-d3d0f03eee22 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:28.800286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23847759-8ba5-4219-a0c2-d3d0f03eee22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:28.845556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-23847759-8ba5-4219-a0c2-d3d0f03eee22 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:28.851615 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-23847759-8ba5-4219-a0c2-d3d0f03eee22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:29.010566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1131/2276] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:29.035023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:29.037312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:29.155158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12172ca8-2cdf-48ed-9940-b4933edf49b1 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:29.163783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69b458-e6df-445a-8ea2-1c588378c79c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:29.164385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69b458-e6df-445a-8ea2-1c588378c79c service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:29.164720 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69b458-e6df-445a-8ea2-1c588378c79c service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:29.165308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69b458-e6df-445a-8ea2-1c588378c79c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:29.185673 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69b458-e6df-445a-8ea2-1c588378c79c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:29.187365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69b458-e6df-445a-8ea2-1c588378c79c service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:29.220160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1146/2277] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:29.938572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23847759-8ba5-4219-a0c2-d3d0f03eee22 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:29.940296 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-23847759-8ba5-4219-a0c2-d3d0f03eee22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:29.963667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23847759-8ba5-4219-a0c2-d3d0f03eee22 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:29.969622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:29.970847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:29.970847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:29.971025 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c admin admin] PATH_INFO: `/v3/projects/c5ee6a6d38bd4801aba2f913fc51838e` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:29.972290 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5ee6a6d38bd4801aba2f913fc51838e)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:29.974396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:29.983895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:30.033500 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5ee6a6d38bd4801aba2f913fc51838e', 'request_id': 'req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:30.046121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b62ca-a6bf-46ec-8952-ffb72c2169ec None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.046673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5ee6a6d38bd4801aba2f913fc51838e', 'request_id': 'req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:30.047153 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b62ca-a6bf-46ec-8952-ffb72c2169ec None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.047608 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b62ca-a6bf-46ec-8952-ffb72c2169ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.052965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5ee6a6d38bd4801aba2f913fc51838e', 'request_id': 'req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:30.073037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c admin admin] The token cache is being invalidate because project c5ee6a6d38bd4801aba2f913fc51838e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:30.073449 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:30.074177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f641371b-7972-4fd7-bf25-a8f6ebda8c0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:30.076465 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1132/2278] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:29 2022] DELETE /identity/v3/projects/c5ee6a6d38bd4801aba2f913fc51838e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:30.081485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0ba80-1edc-4213-93cd-782014689a77 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.081979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0ba80-1edc-4213-93cd-782014689a77 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.082264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0ba80-1edc-4213-93cd-782014689a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.083091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c36b62ca-a6bf-46ec-8952-ffb72c2169ec None None] MFA Rules not processed for user `cd036c73e5c84fa1ba548be4eae80804`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:30.084316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c36b62ca-a6bf-46ec-8952-ffb72c2169ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:30.107197 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-e2d0ba80-1edc-4213-93cd-782014689a77 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:30.109041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e2d0ba80-1edc-4213-93cd-782014689a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:30.168006 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1147/2279] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:48:30.170659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b62ca-a6bf-46ec-8952-ffb72c2169ec None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:30.172327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c36b62ca-a6bf-46ec-8952-ffb72c2169ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:30.185702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1133/2280] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:48:30.249323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b62ca-a6bf-46ec-8952-ffb72c2169ec None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:30.254419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61a66b-3188-4f02-bf14-ab5a61c017d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:30.255259 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e1853-12eb-48e9-bf68-c714196194a5 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.255379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61a66b-3188-4f02-bf14-ab5a61c017d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.255550 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e1853-12eb-48e9-bf68-c714196194a5 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.255830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61a66b-3188-4f02-bf14-ab5a61c017d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.255927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e1853-12eb-48e9-bf68-c714196194a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.256273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61a66b-3188-4f02-bf14-ab5a61c017d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.257136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61a66b-3188-4f02-bf14-ab5a61c017d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:30.259384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61a66b-3188-4f02-bf14-ab5a61c017d1 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:30.275578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:30.280064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-fc4e1853-12eb-48e9-bf68-c714196194a5 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:30.281335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-fc4e1853-12eb-48e9-bf68-c714196194a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:30.292454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1134/2281] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:48:30.296353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e1853-12eb-48e9-bf68-c714196194a5 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:30.306245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1148/2282] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:30 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:48:30.306245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61a66b-3188-4f02-bf14-ab5a61c017d1 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:30.313586 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e1853-12eb-48e9-bf68-c714196194a5 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:30.387115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359819a-5236-4a7d-9409-b026007c0f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:30.390959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359819a-5236-4a7d-9409-b026007c0f83 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.391255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359819a-5236-4a7d-9409-b026007c0f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.391538 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359819a-5236-4a7d-9409-b026007c0f83 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.395668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359819a-5236-4a7d-9409-b026007c0f83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:48:30.396061 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359819a-5236-4a7d-9409-b026007c0f83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:30.397293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61a66b-3188-4f02-bf14-ab5a61c017d1 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:30.400008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359819a-5236-4a7d-9409-b026007c0f83 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:30.406053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88db84d-2d3a-4df6-a5d4-43cc95e3a6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:30.406834 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88db84d-2d3a-4df6-a5d4-43cc95e3a6c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.407236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88db84d-2d3a-4df6-a5d4-43cc95e3a6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.407590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88db84d-2d3a-4df6-a5d4-43cc95e3a6c5 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.408667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88db84d-2d3a-4df6-a5d4-43cc95e3a6c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:30.411300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88db84d-2d3a-4df6-a5d4-43cc95e3a6c5 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:30.416284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1135/2283] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:48:30.420728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359819a-5236-4a7d-9409-b026007c0f83 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:30.435374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359819a-5236-4a7d-9409-b026007c0f83 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:30.443899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1149/2284] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:30 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:30.444039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26904da9-a20c-4370-89cb-c2f23d27823b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:30.444039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26904da9-a20c-4370-89cb-c2f23d27823b admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.444039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26904da9-a20c-4370-89cb-c2f23d27823b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.444039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26904da9-a20c-4370-89cb-c2f23d27823b admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.444385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26904da9-a20c-4370-89cb-c2f23d27823b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:30.445293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26904da9-a20c-4370-89cb-c2f23d27823b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:30.445628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88db84d-2d3a-4df6-a5d4-43cc95e3a6c5 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:30.470194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88db84d-2d3a-4df6-a5d4-43cc95e3a6c5 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:30.475981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156942e6-32ec-4098-a5d4-24bc9419407c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:30.476517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156942e6-32ec-4098-a5d4-24bc9419407c admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.476935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156942e6-32ec-4098-a5d4-24bc9419407c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.477226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156942e6-32ec-4098-a5d4-24bc9419407c admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.478277 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156942e6-32ec-4098-a5d4-24bc9419407c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:30.480028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156942e6-32ec-4098-a5d4-24bc9419407c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:30.484865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1136/2285] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:30 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:30.488641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26904da9-a20c-4370-89cb-c2f23d27823b admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:30.491044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1150/2286] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:30.495381 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156942e6-32ec-4098-a5d4-24bc9419407c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:30.497350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:30.510073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156942e6-32ec-4098-a5d4-24bc9419407c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:30.562437 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26904da9-a20c-4370-89cb-c2f23d27823b admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:30.567415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746ba48-f4d7-4340-a3b2-bc4961a3d00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:30.568099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746ba48-f4d7-4340-a3b2-bc4961a3d00f admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.568380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746ba48-f4d7-4340-a3b2-bc4961a3d00f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.568689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746ba48-f4d7-4340-a3b2-bc4961a3d00f admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.569401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746ba48-f4d7-4340-a3b2-bc4961a3d00f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:30.571328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746ba48-f4d7-4340-a3b2-bc4961a3d00f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:30.596635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1137/2287] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:30 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:48:30.598953 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746ba48-f4d7-4340-a3b2-bc4961a3d00f admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:30.599468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37903a5e-1079-4ea6-8ec2-d683bada4c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:30.600439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37903a5e-1079-4ea6-8ec2-d683bada4c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.600921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37903a5e-1079-4ea6-8ec2-d683bada4c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.601271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37903a5e-1079-4ea6-8ec2-d683bada4c45 admin admin] PATH_INFO: `/v3/projects/a4c21500aa804b7793d171a941d61706/users/0aa05c58ebca42b28d3a15ec222aee76/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.614273 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746ba48-f4d7-4340-a3b2-bc4961a3d00f admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:30.618443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37903a5e-1079-4ea6-8ec2-d683bada4c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4c21500aa804b7793d171a941d61706, user_id=0aa05c58ebca42b28d3a15ec222aee76, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:30.619339 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9240166-299d-44f6-8240-07bb2a9cfdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:30.620016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9240166-299d-44f6-8240-07bb2a9cfdd4 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.620300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9240166-299d-44f6-8240-07bb2a9cfdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.620601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9240166-299d-44f6-8240-07bb2a9cfdd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.621348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9240166-299d-44f6-8240-07bb2a9cfdd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:30.623454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37903a5e-1079-4ea6-8ec2-d683bada4c45 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:30.623570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9240166-299d-44f6-8240-07bb2a9cfdd4 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:30.638156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1138/2288] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:30.643052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1151/2289] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:30 2022] PUT /identity/v3/projects/a4c21500aa804b7793d171a941d61706/users/0aa05c58ebca42b28d3a15ec222aee76/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:30.645604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9240166-299d-44f6-8240-07bb2a9cfdd4 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:30.646869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37903a5e-1079-4ea6-8ec2-d683bada4c45 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:30.660669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9240166-299d-44f6-8240-07bb2a9cfdd4 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:30.727354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:30.728277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb6d50-7e2a-4d53-908d-6a62dea5c164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:30.729941 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb6d50-7e2a-4d53-908d-6a62dea5c164 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.730229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb6d50-7e2a-4d53-908d-6a62dea5c164 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.730862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb6d50-7e2a-4d53-908d-6a62dea5c164 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.731749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb6d50-7e2a-4d53-908d-6a62dea5c164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:30.733618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb6d50-7e2a-4d53-908d-6a62dea5c164 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:30.734902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37903a5e-1079-4ea6-8ec2-d683bada4c45 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:30.741618 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d0c4b-f7fc-496c-8f76-d8b61911c55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:30.742453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d0c4b-f7fc-496c-8f76-d8b61911c55a admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.742747 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d0c4b-f7fc-496c-8f76-d8b61911c55a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.743027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d0c4b-f7fc-496c-8f76-d8b61911c55a admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.743765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d0c4b-f7fc-496c-8f76-d8b61911c55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:30.745479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d0c4b-f7fc-496c-8f76-d8b61911c55a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:30.746201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1139/2290] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:30.750317 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb6d50-7e2a-4d53-908d-6a62dea5c164 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:30.761569 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb6d50-7e2a-4d53-908d-6a62dea5c164 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:30.766189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1152/2291] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:30.767174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de197f-f98c-4616-93e4-3f410596c497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:30.768072 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de197f-f98c-4616-93e4-3f410596c497 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.768361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de197f-f98c-4616-93e4-3f410596c497 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.768679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de197f-f98c-4616-93e4-3f410596c497 admin admin] PATH_INFO: `/v3/projects/00dc4c5138824b8593691320fdab1b12/users/0db5cb4d457c45db8606d8c36c20b4d9/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.770571 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d0c4b-f7fc-496c-8f76-d8b61911c55a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:30.781344 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d0c4b-f7fc-496c-8f76-d8b61911c55a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:30.786805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d524c3-571e-4d97-b698-ea209a609fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:30.787725 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d524c3-571e-4d97-b698-ea209a609fce admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.788026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d524c3-571e-4d97-b698-ea209a609fce admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.788302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d524c3-571e-4d97-b698-ea209a609fce admin admin] PATH_INFO: `/v3/projects/a4c21500aa804b7793d171a941d61706/users/0aa05c58ebca42b28d3a15ec222aee76/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.790400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d524c3-571e-4d97-b698-ea209a609fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4c21500aa804b7793d171a941d61706, user_id=0aa05c58ebca42b28d3a15ec222aee76, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:30.790515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de197f-f98c-4616-93e4-3f410596c497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00dc4c5138824b8593691320fdab1b12, user_id=0db5cb4d457c45db8606d8c36c20b4d9, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:30.793254 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d524c3-571e-4d97-b698-ea209a609fce admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:30.796287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de197f-f98c-4616-93e4-3f410596c497 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:30.805743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1153/2292] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:30 2022] PUT /identity/v3/projects/a4c21500aa804b7793d171a941d61706/users/0aa05c58ebca42b28d3a15ec222aee76/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:30.831686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1140/2293] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:30 2022] PUT /identity/v3/projects/00dc4c5138824b8593691320fdab1b12/users/0db5cb4d457c45db8606d8c36c20b4d9/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:48:30.838539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930e2d7-54cf-4048-9db7-fbaf37ff0019 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:30.838914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930e2d7-54cf-4048-9db7-fbaf37ff0019 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:30.839858 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930e2d7-54cf-4048-9db7-fbaf37ff0019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:30.883992 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4930e2d7-54cf-4048-9db7-fbaf37ff0019 None None] MFA Rules not processed for user `0db5cb4d457c45db8606d8c36c20b4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:30.885347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4930e2d7-54cf-4048-9db7-fbaf37ff0019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:30.968948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1154/2294] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:30.973460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de197f-f98c-4616-93e4-3f410596c497 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:31.047291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:31.061213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de197f-f98c-4616-93e4-3f410596c497 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:31.067897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73590dec-ce0a-4f39-991e-011ed02e3660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:31.069152 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73590dec-ce0a-4f39-991e-011ed02e3660 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:31.069488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73590dec-ce0a-4f39-991e-011ed02e3660 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:31.069767 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73590dec-ce0a-4f39-991e-011ed02e3660 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:31.070488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73590dec-ce0a-4f39-991e-011ed02e3660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:31.072863 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73590dec-ce0a-4f39-991e-011ed02e3660 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:31.126639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1141/2295] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:30 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:48:31.131006 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930e2d7-54cf-4048-9db7-fbaf37ff0019 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:31.149933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930e2d7-54cf-4048-9db7-fbaf37ff0019 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:31.227133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02d4fd-7659-4106-b3d5-1b767d6c7e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:31.227818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02d4fd-7659-4106-b3d5-1b767d6c7e51 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:31.228102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02d4fd-7659-4106-b3d5-1b767d6c7e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:31.228373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02d4fd-7659-4106-b3d5-1b767d6c7e51 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:31.229179 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02d4fd-7659-4106-b3d5-1b767d6c7e51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:31.231172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02d4fd-7659-4106-b3d5-1b767d6c7e51 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:31.270770 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1155/2296] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:31 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:31.274914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73590dec-ce0a-4f39-991e-011ed02e3660 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:31.290365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73590dec-ce0a-4f39-991e-011ed02e3660 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:31.299075 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c6f51-add3-46c5-9358-13a6a63e829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:31.299800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c6f51-add3-46c5-9358-13a6a63e829a admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:31.300086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c6f51-add3-46c5-9358-13a6a63e829a admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:31.300368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c6f51-add3-46c5-9358-13a6a63e829a admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:31.301164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c6f51-add3-46c5-9358-13a6a63e829a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:31.302822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c6f51-add3-46c5-9358-13a6a63e829a admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:31.321772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02d4fd-7659-4106-b3d5-1b767d6c7e51 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:31.322476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1142/2297] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:31.332838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02d4fd-7659-4106-b3d5-1b767d6c7e51 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:31.340267 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085a90e-96d3-41fc-9e30-d77101a99164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:31.342348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085a90e-96d3-41fc-9e30-d77101a99164 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:31.342634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085a90e-96d3-41fc-9e30-d77101a99164 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:31.342913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085a90e-96d3-41fc-9e30-d77101a99164 admin admin] PATH_INFO: `/v3/projects/fedd4d8c8e00421ab7a88ac1ba8359dc/users/0634eb7f583c42de840d26850ef8ff15/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:31.368888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085a90e-96d3-41fc-9e30-d77101a99164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fedd4d8c8e00421ab7a88ac1ba8359dc, user_id=0634eb7f583c42de840d26850ef8ff15, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:31.374105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085a90e-96d3-41fc-9e30-d77101a99164 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:31.396016 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1156/2298] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:31 2022] PUT /identity/v3/projects/fedd4d8c8e00421ab7a88ac1ba8359dc/users/0634eb7f583c42de840d26850ef8ff15/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:31.400254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c6f51-add3-46c5-9358-13a6a63e829a admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:31.412528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c6f51-add3-46c5-9358-13a6a63e829a admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:31.552029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1cccc-ade2-4def-a0e4-a524052cfd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:31.553209 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1cccc-ade2-4def-a0e4-a524052cfd91 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:31.553498 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1cccc-ade2-4def-a0e4-a524052cfd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:31.553775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1cccc-ade2-4def-a0e4-a524052cfd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:31.554885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1cccc-ade2-4def-a0e4-a524052cfd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:31.557204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1cccc-ade2-4def-a0e4-a524052cfd91 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:31.573038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1143/2299] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:31.583292 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085a90e-96d3-41fc-9e30-d77101a99164 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:31.596253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085a90e-96d3-41fc-9e30-d77101a99164 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:31.602014 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac661ed-400f-4fd9-bb14-4f6737f1cc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:31.603208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac661ed-400f-4fd9-bb14-4f6737f1cc0f admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:31.603523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac661ed-400f-4fd9-bb14-4f6737f1cc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:31.603809 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac661ed-400f-4fd9-bb14-4f6737f1cc0f admin admin] PATH_INFO: `/v3/projects/fedd4d8c8e00421ab7a88ac1ba8359dc/users/0634eb7f583c42de840d26850ef8ff15/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:31.606043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac661ed-400f-4fd9-bb14-4f6737f1cc0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fedd4d8c8e00421ab7a88ac1ba8359dc, user_id=0634eb7f583c42de840d26850ef8ff15, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:31.609795 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac661ed-400f-4fd9-bb14-4f6737f1cc0f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:31.624628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1157/2300] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:31 2022] PUT /identity/v3/projects/fedd4d8c8e00421ab7a88ac1ba8359dc/users/0634eb7f583c42de840d26850ef8ff15/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:48:31.636095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874cf63-a9fb-434b-857b-d462a85e47f2 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:31.636411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874cf63-a9fb-434b-857b-d462a85e47f2 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:31.636705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874cf63-a9fb-434b-857b-d462a85e47f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:31.677143 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-7874cf63-a9fb-434b-857b-d462a85e47f2 None None] MFA Rules not processed for user `0634eb7f583c42de840d26850ef8ff15`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:31.678377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7874cf63-a9fb-434b-857b-d462a85e47f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:31.766361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1144/2301] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:31.779925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac661ed-400f-4fd9-bb14-4f6737f1cc0f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:31.783046 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6ac661ed-400f-4fd9-bb14-4f6737f1cc0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:31.822486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:31.886794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac661ed-400f-4fd9-bb14-4f6737f1cc0f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:31.892801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe4f0a-150a-4fed-9271-bf833a84f9ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:31.893595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe4f0a-150a-4fed-9271-bf833a84f9ca service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:31.893950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe4f0a-150a-4fed-9271-bf833a84f9ca service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:31.894293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe4f0a-150a-4fed-9271-bf833a84f9ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:31.980608 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe4f0a-150a-4fed-9271-bf833a84f9ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:31.981131 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe4f0a-150a-4fed-9271-bf833a84f9ca service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:31.992076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1158/2302] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:32.815218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57ef17-a963-43bc-9a42-3bcf734ea70a None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:32.815707 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57ef17-a963-43bc-9a42-3bcf734ea70a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:32.815803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57ef17-a963-43bc-9a42-3bcf734ea70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:32.842790 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ef57ef17-a963-43bc-9a42-3bcf734ea70a None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:32.844694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ef57ef17-a963-43bc-9a42-3bcf734ea70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:32.919256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:32.945457 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1145/2303] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:48:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:33.005443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe4f0a-150a-4fed-9271-bf833a84f9ca service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:33.008006 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4fbe4f0a-150a-4fed-9271-bf833a84f9ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:33.098201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe4f0a-150a-4fed-9271-bf833a84f9ca service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:33.103322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49b6a6-b7e1-433f-a7e0-340b977f582a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:33.103976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49b6a6-b7e1-433f-a7e0-340b977f582a service swift] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:33.104269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49b6a6-b7e1-433f-a7e0-340b977f582a service swift] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:33.104542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49b6a6-b7e1-433f-a7e0-340b977f582a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:33.115653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49b6a6-b7e1-433f-a7e0-340b977f582a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:33.117369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49b6a6-b7e1-433f-a7e0-340b977f582a service swift] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:33.127628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1159/2304] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:48:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:48:33.726824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea1c6c-e266-45bc-9261-b399ab87560e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:33.727824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea1c6c-e266-45bc-9261-b399ab87560e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:33.727824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea1c6c-e266-45bc-9261-b399ab87560e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:33.775205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-90ea1c6c-e266-45bc-9261-b399ab87560e None None] MFA Rules not processed for user `316c65a9dcd8472ab10b532537e5dad7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:33.776509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-90ea1c6c-e266-45bc-9261-b399ab87560e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:33.894104 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1146/2305] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:35.943542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49b6a6-b7e1-433f-a7e0-340b977f582a service swift] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:36.122478 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49b6a6-b7e1-433f-a7e0-340b977f582a service swift] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:36.137588 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:36.138584 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:36.138866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:36.139220 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] PATH_INFO: `/v3/users/0db5cb4d457c45db8606d8c36c20b4d9` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:36.141544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0db5cb4d457c45db8606d8c36c20b4d9)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:36.146003 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:36.162342 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:36.269703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0db5cb4d457c45db8606d8c36c20b4d9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:36.270417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0db5cb4d457c45db8606d8c36c20b4d9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:36.296549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0db5cb4d457c45db8606d8c36c20b4d9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:36.323758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0db5cb4d457c45db8606d8c36c20b4d9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:36.345927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0db5cb4d457c45db8606d8c36c20b4d9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:36.361228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1160/2306] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:35 2022] DELETE /identity/v3/users/0db5cb4d457c45db8606d8c36c20b4d9 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:36.363865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047ba32-5712-4893-ba5f-db6a68e44559 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:36.364425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047ba32-5712-4893-ba5f-db6a68e44559 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:36.364559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047ba32-5712-4893-ba5f-db6a68e44559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:36.423437 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-4047ba32-5712-4893-ba5f-db6a68e44559 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:36.437477 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4047ba32-5712-4893-ba5f-db6a68e44559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:36.742962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1147/2307] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:36.974233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:36.975705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:37.212078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69117aa-f5aa-4509-931f-87cb7429db99 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:37.232732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42def9d-6c70-4f86-8996-ca2793d664a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:37.233410 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42def9d-6c70-4f86-8996-ca2793d664a4 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:37.233697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42def9d-6c70-4f86-8996-ca2793d664a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:37.233974 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42def9d-6c70-4f86-8996-ca2793d664a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:37.251589 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42def9d-6c70-4f86-8996-ca2793d664a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:37.253112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42def9d-6c70-4f86-8996-ca2793d664a4 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:37.300923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1161/2308] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:39.060348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047ba32-5712-4893-ba5f-db6a68e44559 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:39.062382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4047ba32-5712-4893-ba5f-db6a68e44559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:39.085523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047ba32-5712-4893-ba5f-db6a68e44559 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:39.103480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:39.104642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:39.105132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:39.105574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] PATH_INFO: `/v3/projects/00dc4c5138824b8593691320fdab1b12` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:39.120688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00dc4c5138824b8593691320fdab1b12)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:39.128316 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:39.307314 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00dc4c5138824b8593691320fdab1b12', 'request_id': 'req-61adbce0-06b6-4409-b9ef-774d27196560'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:39.324936 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00dc4c5138824b8593691320fdab1b12', 'request_id': 'req-61adbce0-06b6-4409-b9ef-774d27196560'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:39.333886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00dc4c5138824b8593691320fdab1b12', 'request_id': 'req-61adbce0-06b6-4409-b9ef-774d27196560'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:39.354831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] The token cache is being invalidate because project 00dc4c5138824b8593691320fdab1b12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:39.355523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:39.356417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:39.362693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42def9d-6c70-4f86-8996-ca2793d664a4 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:39.373713 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1148/2309] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:39 2022] DELETE /identity/v3/projects/00dc4c5138824b8593691320fdab1b12 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:39.381596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42def9d-6c70-4f86-8996-ca2793d664a4 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:39.493309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91beea3d-5d02-4888-a059-7be12e574427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:39.494708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beea3d-5d02-4888-a059-7be12e574427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:39.495264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beea3d-5d02-4888-a059-7be12e574427 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:39.496051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beea3d-5d02-4888-a059-7be12e574427 admin admin] PATH_INFO: `/v3/users/0634eb7f583c42de840d26850ef8ff15` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:39.498415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91beea3d-5d02-4888-a059-7be12e574427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0634eb7f583c42de840d26850ef8ff15)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:39.501229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91beea3d-5d02-4888-a059-7be12e574427 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:39.561217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:39.613482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-91beea3d-5d02-4888-a059-7be12e574427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0634eb7f583c42de840d26850ef8ff15'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:39.614979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-91beea3d-5d02-4888-a059-7be12e574427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0634eb7f583c42de840d26850ef8ff15'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:39.630440 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-91beea3d-5d02-4888-a059-7be12e574427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0634eb7f583c42de840d26850ef8ff15'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:39.658629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-91beea3d-5d02-4888-a059-7be12e574427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0634eb7f583c42de840d26850ef8ff15'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:39.676154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-91beea3d-5d02-4888-a059-7be12e574427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0634eb7f583c42de840d26850ef8ff15'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:39.689155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1162/2310] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:48:39 2022] DELETE /identity/v3/users/0634eb7f583c42de840d26850ef8ff15 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:40.746961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:40.750876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:40.779820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adbce0-06b6-4409-b9ef-774d27196560 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:40.831352 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:40.963864 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b19f42-cf57-4160-97d4-3ca2c92d32cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:40.964796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b19f42-cf57-4160-97d4-3ca2c92d32cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:40.965204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b19f42-cf57-4160-97d4-3ca2c92d32cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:40.965348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b19f42-cf57-4160-97d4-3ca2c92d32cb admin admin] PATH_INFO: `/v3/projects/fedd4d8c8e00421ab7a88ac1ba8359dc` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:40.977142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b19f42-cf57-4160-97d4-3ca2c92d32cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fedd4d8c8e00421ab7a88ac1ba8359dc)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:40.979101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b19f42-cf57-4160-97d4-3ca2c92d32cb admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:41.122480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-76b19f42-cf57-4160-97d4-3ca2c92d32cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fedd4d8c8e00421ab7a88ac1ba8359dc', 'request_id': 'req-76b19f42-cf57-4160-97d4-3ca2c92d32cb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:41.157954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-76b19f42-cf57-4160-97d4-3ca2c92d32cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fedd4d8c8e00421ab7a88ac1ba8359dc', 'request_id': 'req-76b19f42-cf57-4160-97d4-3ca2c92d32cb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:41.171517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-76b19f42-cf57-4160-97d4-3ca2c92d32cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fedd4d8c8e00421ab7a88ac1ba8359dc', 'request_id': 'req-76b19f42-cf57-4160-97d4-3ca2c92d32cb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:41.210122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-76b19f42-cf57-4160-97d4-3ca2c92d32cb admin admin] The token cache is being invalidate because project fedd4d8c8e00421ab7a88ac1ba8359dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:48:41.210954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-76b19f42-cf57-4160-97d4-3ca2c92d32cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:41.215364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-76b19f42-cf57-4160-97d4-3ca2c92d32cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:48:41.220527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1149/2311] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:48:40 2022] DELETE /identity/v3/projects/fedd4d8c8e00421ab7a88ac1ba8359dc => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:41.226021 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aad6f-074b-4ae1-94b9-dacfc19d52f1 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:41.227270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aad6f-074b-4ae1-94b9-dacfc19d52f1 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:41.227785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aad6f-074b-4ae1-94b9-dacfc19d52f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:41.267091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-510aad6f-074b-4ae1-94b9-dacfc19d52f1 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:41.269127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-510aad6f-074b-4ae1-94b9-dacfc19d52f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:41.445636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fadc5b-9111-4ff8-af85-12b798767412 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:41.445894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1163/2312] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:41.446019 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fadc5b-9111-4ff8-af85-12b798767412 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:41.446197 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fadc5b-9111-4ff8-af85-12b798767412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:41.497819 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-12fadc5b-9111-4ff8-af85-12b798767412 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:41.499308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-12fadc5b-9111-4ff8-af85-12b798767412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:41.513322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1150/2313] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:41.517851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aad6f-074b-4ae1-94b9-dacfc19d52f1 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:41.532288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aad6f-074b-4ae1-94b9-dacfc19d52f1 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:41.557614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75ef64-dedd-4ca8-91d1-6140c93892eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:41.561201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75ef64-dedd-4ca8-91d1-6140c93892eb admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:41.562194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75ef64-dedd-4ca8-91d1-6140c93892eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:41.562705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75ef64-dedd-4ca8-91d1-6140c93892eb admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:41.566224 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75ef64-dedd-4ca8-91d1-6140c93892eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:48:41.569062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75ef64-dedd-4ca8-91d1-6140c93892eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:41.574094 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75ef64-dedd-4ca8-91d1-6140c93892eb admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:41.627328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fadc5b-9111-4ff8-af85-12b798767412 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:41.628698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1164/2314] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:48:41.650213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fadc5b-9111-4ff8-af85-12b798767412 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:41.665677 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00985c04-2557-437f-98f3-a28024c46026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:41.666655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00985c04-2557-437f-98f3-a28024c46026 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:41.667288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00985c04-2557-437f-98f3-a28024c46026 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:41.667288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00985c04-2557-437f-98f3-a28024c46026 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:41.668108 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00985c04-2557-437f-98f3-a28024c46026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:41.671582 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00985c04-2557-437f-98f3-a28024c46026 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:41.773554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1151/2315] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:48:41 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:41.777145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75ef64-dedd-4ca8-91d1-6140c93892eb admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:41.809948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75ef64-dedd-4ca8-91d1-6140c93892eb admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:41.956979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28874aa-6c0f-4ed5-9b2c-98af729ddd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:41.957838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28874aa-6c0f-4ed5-9b2c-98af729ddd8b admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:41.958233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28874aa-6c0f-4ed5-9b2c-98af729ddd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:41.958621 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28874aa-6c0f-4ed5-9b2c-98af729ddd8b admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:41.959469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28874aa-6c0f-4ed5-9b2c-98af729ddd8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:41.967689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28874aa-6c0f-4ed5-9b2c-98af729ddd8b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:42.021198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a271a-a723-4ca6-8b9e-0595b03bf529 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:42.021729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a271a-a723-4ca6-8b9e-0595b03bf529 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:42.022180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a271a-a723-4ca6-8b9e-0595b03bf529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:42.032918 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1165/2316] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:48:41 2022] POST /identity/v3/users => generated 286 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:48:42.035187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28874aa-6c0f-4ed5-9b2c-98af729ddd8b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:42.067434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-491a271a-a723-4ca6-8b9e-0595b03bf529 None None] MFA Rules not processed for user `0aa05c58ebca42b28d3a15ec222aee76`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:42.069005 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-491a271a-a723-4ca6-8b9e-0595b03bf529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:42.073130 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28874aa-6c0f-4ed5-9b2c-98af729ddd8b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:42.092954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21826ce7-d4c4-4d04-9fae-83a88efcf411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:42.100306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826ce7-d4c4-4d04-9fae-83a88efcf411 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:42.100852 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826ce7-d4c4-4d04-9fae-83a88efcf411 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:42.101244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826ce7-d4c4-4d04-9fae-83a88efcf411 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:42.102279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21826ce7-d4c4-4d04-9fae-83a88efcf411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:42.107845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21826ce7-d4c4-4d04-9fae-83a88efcf411 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:42.150203 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1166/2317] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:42.154862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21826ce7-d4c4-4d04-9fae-83a88efcf411 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:42.168532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21826ce7-d4c4-4d04-9fae-83a88efcf411 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:42.176843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1152/2318] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:48:42.185137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77749b2-7cc3-4085-bb55-3dec0d92d92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:42.185938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77749b2-7cc3-4085-bb55-3dec0d92d92e admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:42.186357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77749b2-7cc3-4085-bb55-3dec0d92d92e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:42.186858 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77749b2-7cc3-4085-bb55-3dec0d92d92e admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:42.188362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77749b2-7cc3-4085-bb55-3dec0d92d92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:42.190301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77749b2-7cc3-4085-bb55-3dec0d92d92e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:42.195307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a271a-a723-4ca6-8b9e-0595b03bf529 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:42.197244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-491a271a-a723-4ca6-8b9e-0595b03bf529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:42.197851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:42.208256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1167/2319] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:48:42.209717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77749b2-7cc3-4085-bb55-3dec0d92d92e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:42.224392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:42.231658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77749b2-7cc3-4085-bb55-3dec0d92d92e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:42.241469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d377ef-28ab-47a9-b4e8-db831952dbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:42.243416 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d377ef-28ab-47a9-b4e8-db831952dbc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:42.243813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d377ef-28ab-47a9-b4e8-db831952dbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:42.244330 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d377ef-28ab-47a9-b4e8-db831952dbc5 admin admin] PATH_INFO: `/v3/projects/1824f163532a4ea7aae438626d730adb/users/e15a4fd64d52462d92617f5e0748e532/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:42.280044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d377ef-28ab-47a9-b4e8-db831952dbc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1824f163532a4ea7aae438626d730adb, user_id=e15a4fd64d52462d92617f5e0748e532, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:42.283257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d377ef-28ab-47a9-b4e8-db831952dbc5 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:42.310930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1168/2320] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:48:42 2022] PUT /identity/v3/projects/1824f163532a4ea7aae438626d730adb/users/e15a4fd64d52462d92617f5e0748e532/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:48:42.322517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e816beb-bbe3-4599-b96a-89500e70ce68 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:42.324441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e816beb-bbe3-4599-b96a-89500e70ce68 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:42.326094 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e816beb-bbe3-4599-b96a-89500e70ce68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:42.353780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a271a-a723-4ca6-8b9e-0595b03bf529 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:42.409083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286aee09-9048-4f00-9aea-6e9a0f0eeedc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '925bf2da4df74e86aa84ec62a267d088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:42.409760 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aee09-9048-4f00-9aea-6e9a0f0eeedc service glance] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:42.410065 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aee09-9048-4f00-9aea-6e9a0f0eeedc service glance] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:42.413722 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aee09-9048-4f00-9aea-6e9a0f0eeedc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:42.417085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-286aee09-9048-4f00-9aea-6e9a0f0eeedc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:42.455180 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-1e816beb-bbe3-4599-b96a-89500e70ce68 None None] MFA Rules not processed for user `e15a4fd64d52462d92617f5e0748e532`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:42.458005 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1e816beb-bbe3-4599-b96a-89500e70ce68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:42.512461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286aee09-9048-4f00-9aea-6e9a0f0eeedc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:42.514906 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286aee09-9048-4f00-9aea-6e9a0f0eeedc service glance] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:42.525339 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1153/2321] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:48:42.560196 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1169/2322] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:42.683166 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286aee09-9048-4f00-9aea-6e9a0f0eeedc service glance] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:42.699949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286aee09-9048-4f00-9aea-6e9a0f0eeedc service glance] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:42.705073 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214e8cd-48a0-4a20-aeab-cc5042e35351 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '925bf2da4df74e86aa84ec62a267d088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:42.706295 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214e8cd-48a0-4a20-aeab-cc5042e35351 service glance] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:42.706809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214e8cd-48a0-4a20-aeab-cc5042e35351 service glance] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:42.707325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214e8cd-48a0-4a20-aeab-cc5042e35351 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:42.719444 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214e8cd-48a0-4a20-aeab-cc5042e35351 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:42.720839 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214e8cd-48a0-4a20-aeab-cc5042e35351 service glance] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:42.732743 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1154/2323] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:48:42.870648 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344f095-f4f4-4b59-9cf7-c27171337302 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:42.871049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344f095-f4f4-4b59-9cf7-c27171337302 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:42.871337 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344f095-f4f4-4b59-9cf7-c27171337302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:42.898379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-5344f095-f4f4-4b59-9cf7-c27171337302 None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:42.900416 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-5344f095-f4f4-4b59-9cf7-c27171337302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:42.991326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1170/2324] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:48:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:48:42.999708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214e8cd-48a0-4a20-aeab-cc5042e35351 service glance] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:43.005513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5214e8cd-48a0-4a20-aeab-cc5042e35351 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:43.081306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:43.141308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214e8cd-48a0-4a20-aeab-cc5042e35351 service glance] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:43.160455 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0cb12-65ee-4d66-ac20-00cc12455f54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:43.161571 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0cb12-65ee-4d66-ac20-00cc12455f54 service swift] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:43.161963 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0cb12-65ee-4d66-ac20-00cc12455f54 service swift] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:43.162346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0cb12-65ee-4d66-ac20-00cc12455f54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:43.183113 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0cb12-65ee-4d66-ac20-00cc12455f54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:43.185817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0cb12-65ee-4d66-ac20-00cc12455f54 service swift] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:43.223046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1155/2325] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:48:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:48:43.871183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14847f56-cacc-4a30-9794-d47bc6ae1140 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:43.871659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14847f56-cacc-4a30-9794-d47bc6ae1140 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:43.871785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14847f56-cacc-4a30-9794-d47bc6ae1140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:43.917551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-14847f56-cacc-4a30-9794-d47bc6ae1140 None None] MFA Rules not processed for user `238f78244df546e5a6fc1a4cd1b73288`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:43.918853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-14847f56-cacc-4a30-9794-d47bc6ae1140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:44.073882 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1171/2326] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:53.211973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=11 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:54.062766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:48:56.504740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50a603-9a22-4d72-b834-112b07e9bf39 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:56.505351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50a603-9a22-4d72-b834-112b07e9bf39 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:56.505351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50a603-9a22-4d72-b834-112b07e9bf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:56.549305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5c50a603-9a22-4d72-b834-112b07e9bf39 None None] MFA Rules not processed for user `cd036c73e5c84fa1ba548be4eae80804`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:56.550547 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5c50a603-9a22-4d72-b834-112b07e9bf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:56.573998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1156/2327] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:48:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:59.564174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee368f2a-3ddf-4bcd-9b55-7e89695a140d None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:59.564803 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee368f2a-3ddf-4bcd-9b55-7e89695a140d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:59.564803 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee368f2a-3ddf-4bcd-9b55-7e89695a140d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:59.589533 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-ee368f2a-3ddf-4bcd-9b55-7e89695a140d None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:48:59.590941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ee368f2a-3ddf-4bcd-9b55-7e89695a140d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:59.607475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1172/2328] 10.222.0.52 () {58 vars in 967 bytes} [Mon Jul 25 08:48:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:48:59.620200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50a603-9a22-4d72-b834-112b07e9bf39 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:59.638434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50a603-9a22-4d72-b834-112b07e9bf39 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:48:59.646681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde8c4a-298c-49d0-add1-1e73f99cb775 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:48:59.647560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde8c4a-298c-49d0-add1-1e73f99cb775 service swift] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:48:59.648035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde8c4a-298c-49d0-add1-1e73f99cb775 service swift] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:48:59.648467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde8c4a-298c-49d0-add1-1e73f99cb775 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:48:59.664722 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde8c4a-298c-49d0-add1-1e73f99cb775 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:48:59.666510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde8c4a-298c-49d0-add1-1e73f99cb775 service swift] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:48:59.675919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1157/2329] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:48:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:48:59.946369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee368f2a-3ddf-4bcd-9b55-7e89695a140d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:59.951376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ee368f2a-3ddf-4bcd-9b55-7e89695a140d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:48:59.961405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde8c4a-298c-49d0-add1-1e73f99cb775 service swift] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:48:59.963036 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-fcde8c4a-298c-49d0-add1-1e73f99cb775 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:49:00.047033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee368f2a-3ddf-4bcd-9b55-7e89695a140d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:49:00.057076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735c369-50fd-4ab0-b51a-7bf93ccb53f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:49:00.057943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735c369-50fd-4ab0-b51a-7bf93ccb53f7 service placement] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:00.058331 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735c369-50fd-4ab0-b51a-7bf93ccb53f7 service placement] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:00.058743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735c369-50fd-4ab0-b51a-7bf93ccb53f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:00.060887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c735c369-50fd-4ab0-b51a-7bf93ccb53f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:49:00.065753 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde8c4a-298c-49d0-add1-1e73f99cb775 service swift] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:49:00.071995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96137ebf-4d29-47a1-9964-691a3bcf6d8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:49:00.072988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96137ebf-4d29-47a1-9964-691a3bcf6d8a service placement] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:00.073374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96137ebf-4d29-47a1-9964-691a3bcf6d8a service placement] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:00.073820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96137ebf-4d29-47a1-9964-691a3bcf6d8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:00.075994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735c369-50fd-4ab0-b51a-7bf93ccb53f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:49:00.077970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735c369-50fd-4ab0-b51a-7bf93ccb53f7 service placement] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:49:00.087581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96137ebf-4d29-47a1-9964-691a3bcf6d8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:49:00.090035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96137ebf-4d29-47a1-9964-691a3bcf6d8a service placement] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:49:00.103758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1158/2330] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:49:00.104724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1173/2331] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:48:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:49:01.805035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96137ebf-4d29-47a1-9964-691a3bcf6d8a service placement] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:49:01.806601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-96137ebf-4d29-47a1-9964-691a3bcf6d8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:49:01.928641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96137ebf-4d29-47a1-9964-691a3bcf6d8a service placement] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:49:01.934833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d7fd-a0e2-473c-912c-c34a456c7a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:49:01.935729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d7fd-a0e2-473c-912c-c34a456c7a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:01.936012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d7fd-a0e2-473c-912c-c34a456c7a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:01.936288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d7fd-a0e2-473c-912c-c34a456c7a79 admin admin] PATH_INFO: `/v3/users/bf0c0131372345e6b23260b466ec8ff9` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:01.937544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4d7fd-a0e2-473c-912c-c34a456c7a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf0c0131372345e6b23260b466ec8ff9)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:49:01.939485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4d7fd-a0e2-473c-912c-c34a456c7a79 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:49:01.995219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-dcf4d7fd-a0e2-473c-912c-c34a456c7a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf0c0131372345e6b23260b466ec8ff9'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:49:02.010736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-dcf4d7fd-a0e2-473c-912c-c34a456c7a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf0c0131372345e6b23260b466ec8ff9'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:49:02.034051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-dcf4d7fd-a0e2-473c-912c-c34a456c7a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf0c0131372345e6b23260b466ec8ff9'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:49:02.035574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:49:02.043994 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-dcf4d7fd-a0e2-473c-912c-c34a456c7a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf0c0131372345e6b23260b466ec8ff9'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:49:02.044920 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-dcf4d7fd-a0e2-473c-912c-c34a456c7a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf0c0131372345e6b23260b466ec8ff9'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:49:02.053133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1159/2332] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:49:01 2022] DELETE /identity/v3/users/bf0c0131372345e6b23260b466ec8ff9 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:49:02.295393 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735c369-50fd-4ab0-b51a-7bf93ccb53f7 service placement] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:49:02.326471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735c369-50fd-4ab0-b51a-7bf93ccb53f7 service placement] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:49:02.394735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:49:02.395624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:02.395904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:02.396177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57 admin admin] PATH_INFO: `/v3/projects/d358c71209624c55b7db08cc3435dbcb` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:02.397427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d358c71209624c55b7db08cc3435dbcb)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:49:02.399330 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:49:02.457781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd358c71209624c55b7db08cc3435dbcb', 'request_id': 'req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:49:02.467572 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd358c71209624c55b7db08cc3435dbcb', 'request_id': 'req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:49:02.482918 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd358c71209624c55b7db08cc3435dbcb', 'request_id': 'req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:49:02.505590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57 admin admin] The token cache is being invalidate because project d358c71209624c55b7db08cc3435dbcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:49:02.506046 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:49:02.507000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4c33ebb9-0cac-48ab-be63-d3fa71c5ef57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:49:02.508934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1174/2333] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:49:02 2022] DELETE /identity/v3/projects/d358c71209624c55b7db08cc3435dbcb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:49:02.513825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b914404-7987-4c03-852c-af39369a9547 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:02.514149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b914404-7987-4c03-852c-af39369a9547 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:02.514430 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b914404-7987-4c03-852c-af39369a9547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:02.537001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3b914404-7987-4c03-852c-af39369a9547 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:49:02.538770 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3b914404-7987-4c03-852c-af39369a9547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:49:02.603953 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1160/2334] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:49:02.612537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7480b7-eab3-45e0-88d9-966172e69303 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:02.612856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7480b7-eab3-45e0-88d9-966172e69303 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:02.613136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7480b7-eab3-45e0-88d9-966172e69303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:02.634811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-3d7480b7-eab3-45e0-88d9-966172e69303 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:49:02.636024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3d7480b7-eab3-45e0-88d9-966172e69303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:49:02.644586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1175/2335] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:49:02.648617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b914404-7987-4c03-852c-af39369a9547 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:49:02.658528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b914404-7987-4c03-852c-af39369a9547 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:49:02.676511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357619c-2e4a-4f35-b57b-cfe90139f823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:49:02.677914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357619c-2e4a-4f35-b57b-cfe90139f823 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:02.678196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357619c-2e4a-4f35-b57b-cfe90139f823 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:02.678472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357619c-2e4a-4f35-b57b-cfe90139f823 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:02.686865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357619c-2e4a-4f35-b57b-cfe90139f823 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:49:02.687248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357619c-2e4a-4f35-b57b-cfe90139f823 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:49:02.689795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357619c-2e4a-4f35-b57b-cfe90139f823 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:49:02.706098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1161/2336] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:49:02.710023 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7480b7-eab3-45e0-88d9-966172e69303 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:49:02.720584 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7480b7-eab3-45e0-88d9-966172e69303 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:49:02.729233 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7a4c7-ce9a-4f75-9989-e7a37d1990de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:49:02.730073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7a4c7-ce9a-4f75-9989-e7a37d1990de admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:02.730423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7a4c7-ce9a-4f75-9989-e7a37d1990de admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:02.731230 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7a4c7-ce9a-4f75-9989-e7a37d1990de admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:02.732026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7a4c7-ce9a-4f75-9989-e7a37d1990de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:49:02.734319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7a4c7-ce9a-4f75-9989-e7a37d1990de admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:49:02.740004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:49:02.770940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1176/2337] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:49:02 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:49:02.775680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357619c-2e4a-4f35-b57b-cfe90139f823 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:49:02.787383 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357619c-2e4a-4f35-b57b-cfe90139f823 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:49:02.852871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57851b03-19dd-499c-bf32-08d9239515c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:49:02.853610 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57851b03-19dd-499c-bf32-08d9239515c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:02.853950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57851b03-19dd-499c-bf32-08d9239515c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:02.854302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57851b03-19dd-499c-bf32-08d9239515c8 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:02.855083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57851b03-19dd-499c-bf32-08d9239515c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:49:02.858417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57851b03-19dd-499c-bf32-08d9239515c8 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:49:02.884221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1162/2338] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:49:02 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:49:02.888153 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7a4c7-ce9a-4f75-9989-e7a37d1990de admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:49:02.899412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7a4c7-ce9a-4f75-9989-e7a37d1990de admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:49:02.919475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e2a1-07e6-4923-861a-558a9f2a158e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:49:02.921431 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e2a1-07e6-4923-861a-558a9f2a158e admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:02.921725 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e2a1-07e6-4923-861a-558a9f2a158e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:02.922003 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e2a1-07e6-4923-861a-558a9f2a158e admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:02.922813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e2a1-07e6-4923-861a-558a9f2a158e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:49:02.924659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e2a1-07e6-4923-861a-558a9f2a158e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:49:02.941539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1177/2339] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:49:02.942203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57851b03-19dd-499c-bf32-08d9239515c8 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:49:02.952901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57851b03-19dd-499c-bf32-08d9239515c8 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:49:02.968246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0d28b-c8b2-4594-a381-b75c83138f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:49:02.969200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0d28b-c8b2-4594-a381-b75c83138f40 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:02.969485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0d28b-c8b2-4594-a381-b75c83138f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:02.969757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0d28b-c8b2-4594-a381-b75c83138f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:02.970503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0d28b-c8b2-4594-a381-b75c83138f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:49:02.972554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0d28b-c8b2-4594-a381-b75c83138f40 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:49:02.992328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1163/2340] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:49:02.996653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e2a1-07e6-4923-861a-558a9f2a158e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:49:03.007226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e2a1-07e6-4923-861a-558a9f2a158e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:49:03.011951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0a2e9-ccc8-4173-acf8-568cb9181360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:49:03.012903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0a2e9-ccc8-4173-acf8-568cb9181360 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:03.013188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0a2e9-ccc8-4173-acf8-568cb9181360 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:03.013600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0a2e9-ccc8-4173-acf8-568cb9181360 admin admin] PATH_INFO: `/v3/projects/d63f1e7e45df467f816bc14d6f861934/users/28a6dd88b54d47bb83bedb2ef86b728b/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:03.028593 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0a2e9-ccc8-4173-acf8-568cb9181360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d63f1e7e45df467f816bc14d6f861934, user_id=28a6dd88b54d47bb83bedb2ef86b728b, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:49:03.031243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0a2e9-ccc8-4173-acf8-568cb9181360 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:49:03.059124 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1178/2341] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:49:02 2022] PUT /identity/v3/projects/d63f1e7e45df467f816bc14d6f861934/users/28a6dd88b54d47bb83bedb2ef86b728b/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:49:03.070759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072cca8-b502-40e3-84c0-6a0029727e0f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:03.071075 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072cca8-b502-40e3-84c0-6a0029727e0f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:03.071424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072cca8-b502-40e3-84c0-6a0029727e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:03.103377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5072cca8-b502-40e3-84c0-6a0029727e0f None None] MFA Rules not processed for user `28a6dd88b54d47bb83bedb2ef86b728b`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:49:03.104613 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5072cca8-b502-40e3-84c0-6a0029727e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:49:03.155145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:49:03.177872 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1164/2342] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:49:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:49:03.189117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0a2e9-ccc8-4173-acf8-568cb9181360 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:49:03.190956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-68b0a2e9-ccc8-4173-acf8-568cb9181360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:49:03.270596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0a2e9-ccc8-4173-acf8-568cb9181360 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:49:03.274345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af30d8e-79f9-4ed9-ac23-372f8a1acb71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:49:03.275007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af30d8e-79f9-4ed9-ac23-372f8a1acb71 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:03.275296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af30d8e-79f9-4ed9-ac23-372f8a1acb71 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:03.275577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af30d8e-79f9-4ed9-ac23-372f8a1acb71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:03.286146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af30d8e-79f9-4ed9-ac23-372f8a1acb71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:49:03.287564 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af30d8e-79f9-4ed9-ac23-372f8a1acb71 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:49:03.313068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1179/2343] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:49:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:49:04.905182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1622631-3c0e-4b0c-9b8c-606fbb0984fa None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:04.905772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1622631-3c0e-4b0c-9b8c-606fbb0984fa None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:04.905865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1622631-3c0e-4b0c-9b8c-606fbb0984fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:04.936894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-b1622631-3c0e-4b0c-9b8c-606fbb0984fa None None] MFA Rules not processed for user `28a6dd88b54d47bb83bedb2ef86b728b`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:49:04.939178 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b1622631-3c0e-4b0c-9b8c-606fbb0984fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:49:04.955215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1165/2344] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:49:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:49:12.582728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d45a3-b364-42f7-bb1e-3add0b3b9c63 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:12.583275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d45a3-b364-42f7-bb1e-3add0b3b9c63 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:12.583275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d45a3-b364-42f7-bb1e-3add0b3b9c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:12.603285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-5f2d45a3-b364-42f7-bb1e-3add0b3b9c63 None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:49:12.605027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-5f2d45a3-b364-42f7-bb1e-3add0b3b9c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:49:12.666221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1180/2345] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:49:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:49:12.678669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1622631-3c0e-4b0c-9b8c-606fbb0984fa None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:49:12.680237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b1622631-3c0e-4b0c-9b8c-606fbb0984fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:49:12.759518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1622631-3c0e-4b0c-9b8c-606fbb0984fa None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:49:12.765449 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e414e-a3a6-471e-9291-a42339dcdc42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:49:12.766187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e414e-a3a6-471e-9291-a42339dcdc42 service swift] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:49:12.766549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e414e-a3a6-471e-9291-a42339dcdc42 service swift] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:49:12.766901 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e414e-a3a6-471e-9291-a42339dcdc42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:49:12.775699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e414e-a3a6-471e-9291-a42339dcdc42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:49:12.777521 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e414e-a3a6-471e-9291-a42339dcdc42 service swift] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:49:12.786997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1166/2346] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:49:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:49:22.659201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:49:22.783435 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=42 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:50:01.792265 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d45a3-b364-42f7-bb1e-3add0b3b9c63 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:01.793743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-5f2d45a3-b364-42f7-bb1e-3add0b3b9c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:01.873183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d45a3-b364-42f7-bb1e-3add0b3b9c63 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:01.878356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:01.879542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:01.880246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:01.880734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] PATH_INFO: `/v3/users/e15a4fd64d52462d92617f5e0748e532` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:01.882852 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e15a4fd64d52462d92617f5e0748e532)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:01.885249 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:01.971227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e15a4fd64d52462d92617f5e0748e532'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:01.971998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e15a4fd64d52462d92617f5e0748e532'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:01.983545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e15a4fd64d52462d92617f5e0748e532'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:02.013853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e15a4fd64d52462d92617f5e0748e532'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:02.027020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e15a4fd64d52462d92617f5e0748e532'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:02.046024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1181/2347] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:50:01 2022] DELETE /identity/v3/users/e15a4fd64d52462d92617f5e0748e532 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:50:02.050925 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1834e8e-7858-41e8-9f34-346e317fcd3b None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:02.051565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1834e8e-7858-41e8-9f34-346e317fcd3b None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:02.051565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1834e8e-7858-41e8-9f34-346e317fcd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:02.083511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-f1834e8e-7858-41e8-9f34-346e317fcd3b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:50:02.084786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f1834e8e-7858-41e8-9f34-346e317fcd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:02.161807 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1167/2348] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:50:02.174887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:02.177349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:02.272675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ad933-af3a-4853-b30a-538e45e5921c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:02.277186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889f0da-68d5-4af0-ad1c-4eb3643a3bf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:02.277934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889f0da-68d5-4af0-ad1c-4eb3643a3bf0 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:02.278224 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889f0da-68d5-4af0-ad1c-4eb3643a3bf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:02.278504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889f0da-68d5-4af0-ad1c-4eb3643a3bf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:02.293032 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889f0da-68d5-4af0-ad1c-4eb3643a3bf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:02.295583 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889f0da-68d5-4af0-ad1c-4eb3643a3bf0 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:02.315682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1182/2349] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:50:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:50:02.872960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1834e8e-7858-41e8-9f34-346e317fcd3b None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:02.874600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f1834e8e-7858-41e8-9f34-346e317fcd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:02.893316 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1834e8e-7858-41e8-9f34-346e317fcd3b None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:02.898476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ce344-c076-4b4f-82bd-4c8b557049c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:02.899462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ce344-c076-4b4f-82bd-4c8b557049c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:02.899853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ce344-c076-4b4f-82bd-4c8b557049c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:02.900238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ce344-c076-4b4f-82bd-4c8b557049c8 admin admin] PATH_INFO: `/v3/projects/1824f163532a4ea7aae438626d730adb` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:02.901835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ce344-c076-4b4f-82bd-4c8b557049c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1824f163532a4ea7aae438626d730adb)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:02.904970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ce344-c076-4b4f-82bd-4c8b557049c8 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:02.981169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-eb6ce344-c076-4b4f-82bd-4c8b557049c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1824f163532a4ea7aae438626d730adb', 'request_id': 'req-eb6ce344-c076-4b4f-82bd-4c8b557049c8'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:02.992756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-eb6ce344-c076-4b4f-82bd-4c8b557049c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1824f163532a4ea7aae438626d730adb', 'request_id': 'req-eb6ce344-c076-4b4f-82bd-4c8b557049c8'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:03.001909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-eb6ce344-c076-4b4f-82bd-4c8b557049c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1824f163532a4ea7aae438626d730adb', 'request_id': 'req-eb6ce344-c076-4b4f-82bd-4c8b557049c8'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:03.049332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-eb6ce344-c076-4b4f-82bd-4c8b557049c8 admin admin] The token cache is being invalidate because project 1824f163532a4ea7aae438626d730adb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:50:03.049706 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-eb6ce344-c076-4b4f-82bd-4c8b557049c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:03.050683 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-eb6ce344-c076-4b4f-82bd-4c8b557049c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:03.052826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1168/2350] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:50:02 2022] DELETE /identity/v3/projects/1824f163532a4ea7aae438626d730adb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:50:12.308051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:50:13.034056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:50:19.245053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526ac72-a003-49f4-bb02-6d0ffcdf5521 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:19.245480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526ac72-a003-49f4-bb02-6d0ffcdf5521 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:19.245623 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526ac72-a003-49f4-bb02-6d0ffcdf5521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:19.265818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-1526ac72-a003-49f4-bb02-6d0ffcdf5521 None None] MFA Rules not processed for user `8e682232b7374450a7265e092602a5a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:50:19.267056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1526ac72-a003-49f4-bb02-6d0ffcdf5521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:19.326329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1183/2351] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:50:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:50:19.645559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615f85a-15e0-45cd-b35f-a3febf57cb18 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:19.646061 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615f85a-15e0-45cd-b35f-a3febf57cb18 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:19.646158 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615f85a-15e0-45cd-b35f-a3febf57cb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:19.664935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-f615f85a-15e0-45cd-b35f-a3febf57cb18 None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:50:19.666147 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f615f85a-15e0-45cd-b35f-a3febf57cb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:19.749428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1169/2352] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:50:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:50:19.799634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526ac72-a003-49f4-bb02-6d0ffcdf5521 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:19.801097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1526ac72-a003-49f4-bb02-6d0ffcdf5521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:19.870745 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526ac72-a003-49f4-bb02-6d0ffcdf5521 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:19.876521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32cdd5-7d29-43c1-95e9-3029ba70a37b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:19.877200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32cdd5-7d29-43c1-95e9-3029ba70a37b service swift] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:19.877485 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32cdd5-7d29-43c1-95e9-3029ba70a37b service swift] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:19.877765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32cdd5-7d29-43c1-95e9-3029ba70a37b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:19.886205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32cdd5-7d29-43c1-95e9-3029ba70a37b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:19.887797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32cdd5-7d29-43c1-95e9-3029ba70a37b service swift] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:19.896682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1184/2353] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:50:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:50:20.512322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32621143-50e7-4963-ae2e-35c592ea87b3 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:20.512652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32621143-50e7-4963-ae2e-35c592ea87b3 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:20.512991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32621143-50e7-4963-ae2e-35c592ea87b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:20.532788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-32621143-50e7-4963-ae2e-35c592ea87b3 None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:50:20.534138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-32621143-50e7-4963-ae2e-35c592ea87b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:20.541356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1170/2354] 10.222.0.52 () {58 vars in 967 bytes} [Mon Jul 25 08:50:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:50:20.553932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32cdd5-7d29-43c1-95e9-3029ba70a37b service swift] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:20.565135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32cdd5-7d29-43c1-95e9-3029ba70a37b service swift] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:20.570305 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282227e-c521-449d-96b9-a8f84850828e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:20.570979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282227e-c521-449d-96b9-a8f84850828e service swift] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:20.571263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282227e-c521-449d-96b9-a8f84850828e service swift] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:20.571543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282227e-c521-449d-96b9-a8f84850828e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:20.580593 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282227e-c521-449d-96b9-a8f84850828e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:20.582206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282227e-c521-449d-96b9-a8f84850828e service swift] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:20.590889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1185/2355] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:50:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:50:20.813804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf61cb1-f31a-4f21-bd9a-b50391cc2239 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:20.814294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf61cb1-f31a-4f21-bd9a-b50391cc2239 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:20.814388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf61cb1-f31a-4f21-bd9a-b50391cc2239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:20.834087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ddf61cb1-f31a-4f21-bd9a-b50391cc2239 None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:50:20.835270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ddf61cb1-f31a-4f21-bd9a-b50391cc2239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:20.842661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1171/2356] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:50:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:50:20.854881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282227e-c521-449d-96b9-a8f84850828e service swift] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:20.865905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282227e-c521-449d-96b9-a8f84850828e service swift] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:20.870869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0366e1c-fb79-4da4-8daa-4ac9c2e94e75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:20.871499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0366e1c-fb79-4da4-8daa-4ac9c2e94e75 service swift] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:20.871784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0366e1c-fb79-4da4-8daa-4ac9c2e94e75 service swift] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:20.872062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0366e1c-fb79-4da4-8daa-4ac9c2e94e75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:20.880989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0366e1c-fb79-4da4-8daa-4ac9c2e94e75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:20.882547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0366e1c-fb79-4da4-8daa-4ac9c2e94e75 service swift] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:20.890542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1186/2357] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:50:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:50:21.239001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa1537-e594-41ba-9e79-5aee5bacddab None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:21.239361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa1537-e594-41ba-9e79-5aee5bacddab None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:21.239716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa1537-e594-41ba-9e79-5aee5bacddab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:21.259482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-f4fa1537-e594-41ba-9e79-5aee5bacddab None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:50:21.260719 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f4fa1537-e594-41ba-9e79-5aee5bacddab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:21.269024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1172/2358] 10.222.0.52 () {58 vars in 967 bytes} [Mon Jul 25 08:50:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:50:21.282099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0366e1c-fb79-4da4-8daa-4ac9c2e94e75 service swift] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:21.293690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0366e1c-fb79-4da4-8daa-4ac9c2e94e75 service swift] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:21.300231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c723ea-6093-4f09-80e3-3772baccb630 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:21.301200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c723ea-6093-4f09-80e3-3772baccb630 service swift] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:21.301652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c723ea-6093-4f09-80e3-3772baccb630 service swift] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:21.302101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c723ea-6093-4f09-80e3-3772baccb630 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:21.312226 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c723ea-6093-4f09-80e3-3772baccb630 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:21.314969 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c723ea-6093-4f09-80e3-3772baccb630 service swift] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:21.323609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1187/2359] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:50:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:50:23.507385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa1537-e594-41ba-9e79-5aee5bacddab None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:23.508842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f4fa1537-e594-41ba-9e79-5aee5bacddab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:23.580508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa1537-e594-41ba-9e79-5aee5bacddab None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:23.585269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:23.586148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:23.586433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:23.586709 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] PATH_INFO: `/v3/users/28a6dd88b54d47bb83bedb2ef86b728b` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:23.587920 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] RBAC: Authorizing `identity:delete_user(user_id=28a6dd88b54d47bb83bedb2ef86b728b)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:23.589876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:23.758129 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28a6dd88b54d47bb83bedb2ef86b728b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:23.765084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28a6dd88b54d47bb83bedb2ef86b728b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:23.777066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28a6dd88b54d47bb83bedb2ef86b728b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:23.784239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28a6dd88b54d47bb83bedb2ef86b728b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:23.784892 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28a6dd88b54d47bb83bedb2ef86b728b'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:23.790612 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1173/2360] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:50:23 2022] DELETE /identity/v3/users/28a6dd88b54d47bb83bedb2ef86b728b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:50:23.795103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac78f5-ad6d-45fa-bec4-d5089cbd3619 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:23.795600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac78f5-ad6d-45fa-bec4-d5089cbd3619 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:23.795728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac78f5-ad6d-45fa-bec4-d5089cbd3619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:23.814668 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7bac78f5-ad6d-45fa-bec4-d5089cbd3619 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:50:23.815797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7bac78f5-ad6d-45fa-bec4-d5089cbd3619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:23.862968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:50:23.874299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1188/2361] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:50:23.883997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:23.885440 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:23.892313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:50:23.950250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add69177-bd91-4d74-b6b0-f6fde87a394f admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:23.954874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f55f13-93cd-489f-9d61-0d8e603d6a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:23.955680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f55f13-93cd-489f-9d61-0d8e603d6a2d service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:23.956058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f55f13-93cd-489f-9d61-0d8e603d6a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:23.956432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f55f13-93cd-489f-9d61-0d8e603d6a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:23.966940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f55f13-93cd-489f-9d61-0d8e603d6a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:23.968378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f55f13-93cd-489f-9d61-0d8e603d6a2d service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:23.979811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1174/2362] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:50:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:50:24.159612 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac78f5-ad6d-45fa-bec4-d5089cbd3619 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:24.161075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7bac78f5-ad6d-45fa-bec4-d5089cbd3619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:24.173216 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac78f5-ad6d-45fa-bec4-d5089cbd3619 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:24.177900 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:24.178767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:24.179053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:24.179328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] PATH_INFO: `/v3/projects/d63f1e7e45df467f816bc14d6f861934` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:24.180544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d63f1e7e45df467f816bc14d6f861934)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:24.182498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:24.230275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd63f1e7e45df467f816bc14d6f861934', 'request_id': 'req-35f0a637-b14c-4877-9460-24b48bc7e949'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:24.237129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd63f1e7e45df467f816bc14d6f861934', 'request_id': 'req-35f0a637-b14c-4877-9460-24b48bc7e949'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:24.242683 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd63f1e7e45df467f816bc14d6f861934', 'request_id': 'req-35f0a637-b14c-4877-9460-24b48bc7e949'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:24.254503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] The token cache is being invalidate because project d63f1e7e45df467f816bc14d6f861934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:50:24.254788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:24.255406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:50:24.256953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1189/2363] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:50:24 2022] DELETE /identity/v3/projects/d63f1e7e45df467f816bc14d6f861934 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:50:33.975239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=24 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:50:34.251330 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:50:44.494245 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f55f13-93cd-489f-9d61-0d8e603d6a2d service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:44.495634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-81f55f13-93cd-489f-9d61-0d8e603d6a2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:44.515286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f55f13-93cd-489f-9d61-0d8e603d6a2d service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:44.563402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55e91b-b9ea-4323-934f-e4b0a72b8651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:44.563975 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e91b-b9ea-4323-934f-e4b0a72b8651 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:44.564256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e91b-b9ea-4323-934f-e4b0a72b8651 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:44.564534 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e91b-b9ea-4323-934f-e4b0a72b8651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:44.566341 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ff55e91b-b9ea-4323-934f-e4b0a72b8651 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:44.634429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55e91b-b9ea-4323-934f-e4b0a72b8651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:44.635839 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55e91b-b9ea-4323-934f-e4b0a72b8651 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:44.647053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1175/2364] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:50:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:50:46.497206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:46.498952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:46.521011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0a637-b14c-4877-9460-24b48bc7e949 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:46.524827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6948f-4ec6-4bb4-ac78-983fe28556f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:46.525478 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6948f-4ec6-4bb4-ac78-983fe28556f9 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:46.525763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6948f-4ec6-4bb4-ac78-983fe28556f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:46.526045 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6948f-4ec6-4bb4-ac78-983fe28556f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:46.527863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4ef6948f-4ec6-4bb4-ac78-983fe28556f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:46.591505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6948f-4ec6-4bb4-ac78-983fe28556f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:46.592971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6948f-4ec6-4bb4-ac78-983fe28556f9 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:46.606272 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1190/2365] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:50:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:50:46.989940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55e91b-b9ea-4323-934f-e4b0a72b8651 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:50:46.998804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55e91b-b9ea-4323-934f-e4b0a72b8651 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:50:47.004727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71656cd9-5953-431f-9b99-b4d6c15e728e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:50:47.005222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71656cd9-5953-431f-9b99-b4d6c15e728e service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:50:47.005507 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71656cd9-5953-431f-9b99-b4d6c15e728e service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:50:47.005784 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71656cd9-5953-431f-9b99-b4d6c15e728e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:50:47.007573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-71656cd9-5953-431f-9b99-b4d6c15e728e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:50:47.021448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71656cd9-5953-431f-9b99-b4d6c15e728e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:50:47.022840 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71656cd9-5953-431f-9b99-b4d6c15e728e service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:50:47.033964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1176/2366] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:50:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:50:56.598355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=30 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:50:57.028949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=60 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:51:07.881971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdc957-7174-4ddf-ba49-6bd3fdfa3b1e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:51:07.882526 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdc957-7174-4ddf-ba49-6bd3fdfa3b1e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:51:07.882526 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdc957-7174-4ddf-ba49-6bd3fdfa3b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:51:07.904015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-b1cdc957-7174-4ddf-ba49-6bd3fdfa3b1e None None] MFA Rules not processed for user `316c65a9dcd8472ab10b532537e5dad7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:51:07.905276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b1cdc957-7174-4ddf-ba49-6bd3fdfa3b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:51:07.962347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1191/2367] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:51:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:51:08.199321 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71656cd9-5953-431f-9b99-b4d6c15e728e service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:51:08.208518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71656cd9-5953-431f-9b99-b4d6c15e728e service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:51:08.212136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77073315-bd05-409c-b4e5-b740ad983a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:51:08.212806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77073315-bd05-409c-b4e5-b740ad983a3e service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:51:08.213089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77073315-bd05-409c-b4e5-b740ad983a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:51:08.213365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77073315-bd05-409c-b4e5-b740ad983a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:51:08.215135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-77073315-bd05-409c-b4e5-b740ad983a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:51:08.225326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77073315-bd05-409c-b4e5-b740ad983a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:51:08.226737 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77073315-bd05-409c-b4e5-b740ad983a3e service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:51:08.235097 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1177/2368] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:51:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:51:08.550204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cdc957-7174-4ddf-ba49-6bd3fdfa3b1e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:51:08.551645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b1cdc957-7174-4ddf-ba49-6bd3fdfa3b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:51:08.634596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cdc957-7174-4ddf-ba49-6bd3fdfa3b1e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:51:08.639928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f969f9-667f-4be7-b398-80ea5e3f221a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:51:08.640652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f969f9-667f-4be7-b398-80ea5e3f221a service placement] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:51:08.640944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f969f9-667f-4be7-b398-80ea5e3f221a service placement] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:51:08.641225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f969f9-667f-4be7-b398-80ea5e3f221a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:51:08.643063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a0f969f9-667f-4be7-b398-80ea5e3f221a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:51:08.654734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f969f9-667f-4be7-b398-80ea5e3f221a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:51:08.656336 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f969f9-667f-4be7-b398-80ea5e3f221a service placement] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:51:08.666785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1192/2369] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:51:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:51:10.375959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77073315-bd05-409c-b4e5-b740ad983a3e service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:51:10.377414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-77073315-bd05-409c-b4e5-b740ad983a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:51:10.387800 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77073315-bd05-409c-b4e5-b740ad983a3e service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:51:10.391383 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe744c-c703-4656-a721-7c0ffab085d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:51:10.392020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe744c-c703-4656-a721-7c0ffab085d8 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:51:10.392297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe744c-c703-4656-a721-7c0ffab085d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:51:10.392588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe744c-c703-4656-a721-7c0ffab085d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:51:10.394309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f0fe744c-c703-4656-a721-7c0ffab085d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:51:10.403577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe744c-c703-4656-a721-7c0ffab085d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:51:10.405040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe744c-c703-4656-a721-7c0ffab085d8 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:51:10.413291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1178/2370] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:51:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:51:18.662242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=57 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:51:20.409458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=12 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:51:51.847251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f969f9-667f-4be7-b398-80ea5e3f221a service placement] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:51:51.849851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a0f969f9-667f-4be7-b398-80ea5e3f221a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:51:51.890253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f969f9-667f-4be7-b398-80ea5e3f221a service placement] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:51:51.900341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3a822-06e8-4962-b039-beb5deaf1b75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:51:51.901549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3a822-06e8-4962-b039-beb5deaf1b75 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:51:51.901953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3a822-06e8-4962-b039-beb5deaf1b75 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:51:51.902379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3a822-06e8-4962-b039-beb5deaf1b75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:51:51.906997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8cc3a822-06e8-4962-b039-beb5deaf1b75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:51:51.923549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3a822-06e8-4962-b039-beb5deaf1b75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:51:51.925438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3a822-06e8-4962-b039-beb5deaf1b75 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:51:51.937271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1193/2371] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:51:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:52:01.932395 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=6 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:52:05.155408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe744c-c703-4656-a721-7c0ffab085d8 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:05.178509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe744c-c703-4656-a721-7c0ffab085d8 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:05.185664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b14de3-8df8-407c-b60a-f1dc318099ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:05.186333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b14de3-8df8-407c-b60a-f1dc318099ae service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:05.186621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b14de3-8df8-407c-b60a-f1dc318099ae service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:05.186895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b14de3-8df8-407c-b60a-f1dc318099ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:05.188797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a9b14de3-8df8-407c-b60a-f1dc318099ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:05.205586 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b14de3-8df8-407c-b60a-f1dc318099ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:05.207178 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b14de3-8df8-407c-b60a-f1dc318099ae service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:05.219123 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1179/2372] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:52:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:52:06.862858 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503809f-1a41-4ba4-8a4f-edf29209f387 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:06.863440 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503809f-1a41-4ba4-8a4f-edf29209f387 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:06.863537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503809f-1a41-4ba4-8a4f-edf29209f387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:06.884708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-3503809f-1a41-4ba4-8a4f-edf29209f387 None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:52:06.885938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3503809f-1a41-4ba4-8a4f-edf29209f387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:06.951007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1194/2373] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:52:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:52:07.000533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b14de3-8df8-407c-b60a-f1dc318099ae service nova] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:07.002263 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a9b14de3-8df8-407c-b60a-f1dc318099ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:07.080164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b14de3-8df8-407c-b60a-f1dc318099ae service nova] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:07.085861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de12744-3a23-484b-9108-1daf176f1828 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:07.086534 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de12744-3a23-484b-9108-1daf176f1828 service swift] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:07.086824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de12744-3a23-484b-9108-1daf176f1828 service swift] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:07.087106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de12744-3a23-484b-9108-1daf176f1828 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:07.095965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de12744-3a23-484b-9108-1daf176f1828 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:07.097795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de12744-3a23-484b-9108-1daf176f1828 service swift] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:07.107318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1180/2374] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:52:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:52:16.942948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=25 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:52:17.103322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=38 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:52:26.445213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdc133-a80e-451a-8067-c91dc1d76b6e None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:26.445816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdc133-a80e-451a-8067-c91dc1d76b6e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:26.445816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdc133-a80e-451a-8067-c91dc1d76b6e None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:26.447182 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1195/2375] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:52:26.481894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3e458-be98-460a-9fb0-23fa40547dfa None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:26.482509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3e458-be98-460a-9fb0-23fa40547dfa None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:26.482509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3e458-be98-460a-9fb0-23fa40547dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:26.504797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-b9d3e458-be98-460a-9fb0-23fa40547dfa None None] MFA Rules not processed for user `62bf4ef4fce74f7b91e453f73430d9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:52:26.506323 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b9d3e458-be98-460a-9fb0-23fa40547dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:26.515566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1181/2376] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:52:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:52:26.546065 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdc133-a80e-451a-8067-c91dc1d76b6e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:26.548201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d2cdc133-a80e-451a-8067-c91dc1d76b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:26.567512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdc133-a80e-451a-8067-c91dc1d76b6e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:26.572689 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc7566-fd50-424f-99e6-8e22bfa363f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:26.573356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc7566-fd50-424f-99e6-8e22bfa363f0 service placement] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:26.573653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc7566-fd50-424f-99e6-8e22bfa363f0 service placement] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:26.573915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc7566-fd50-424f-99e6-8e22bfa363f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:26.583288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc7566-fd50-424f-99e6-8e22bfa363f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:26.586008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc7566-fd50-424f-99e6-8e22bfa363f0 service placement] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:26.587138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3e458-be98-460a-9fb0-23fa40547dfa None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:26.588692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b9d3e458-be98-460a-9fb0-23fa40547dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:26.596375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1196/2377] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:52:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:52:26.663196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3e458-be98-460a-9fb0-23fa40547dfa None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:26.668474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be6e6b-07d2-4e61-a9fc-4d7d599ac1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:26.669395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be6e6b-07d2-4e61-a9fc-4d7d599ac1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:26.669686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be6e6b-07d2-4e61-a9fc-4d7d599ac1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:26.669967 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be6e6b-07d2-4e61-a9fc-4d7d599ac1cc admin admin] PATH_INFO: `/v3/users/316c65a9dcd8472ab10b532537e5dad7` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:26.671313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78be6e6b-07d2-4e61-a9fc-4d7d599ac1cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=316c65a9dcd8472ab10b532537e5dad7)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:26.673305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78be6e6b-07d2-4e61-a9fc-4d7d599ac1cc admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:26.724223 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-78be6e6b-07d2-4e61-a9fc-4d7d599ac1cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '316c65a9dcd8472ab10b532537e5dad7'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:26.735897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-78be6e6b-07d2-4e61-a9fc-4d7d599ac1cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '316c65a9dcd8472ab10b532537e5dad7'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:26.762677 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-78be6e6b-07d2-4e61-a9fc-4d7d599ac1cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '316c65a9dcd8472ab10b532537e5dad7'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:26.770499 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-78be6e6b-07d2-4e61-a9fc-4d7d599ac1cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '316c65a9dcd8472ab10b532537e5dad7'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:26.771261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-78be6e6b-07d2-4e61-a9fc-4d7d599ac1cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '316c65a9dcd8472ab10b532537e5dad7'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:26.778119 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1182/2378] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:52:26 2022] DELETE /identity/v3/users/316c65a9dcd8472ab10b532537e5dad7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:52:26.969948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc7566-fd50-424f-99e6-8e22bfa363f0 service placement] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:26.984106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc7566-fd50-424f-99e6-8e22bfa363f0 service placement] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:27.046856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1493cf0-6df3-45ba-8903-5d3f7d707141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:27.047828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1493cf0-6df3-45ba-8903-5d3f7d707141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.048339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1493cf0-6df3-45ba-8903-5d3f7d707141 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.048716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1493cf0-6df3-45ba-8903-5d3f7d707141 admin admin] PATH_INFO: `/v3/projects/1e76bc803d784442936b25a7ad36d564` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.050359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1493cf0-6df3-45ba-8903-5d3f7d707141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e76bc803d784442936b25a7ad36d564)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:27.052274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1493cf0-6df3-45ba-8903-5d3f7d707141 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:27.113301 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c1493cf0-6df3-45ba-8903-5d3f7d707141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e76bc803d784442936b25a7ad36d564', 'request_id': 'req-c1493cf0-6df3-45ba-8903-5d3f7d707141'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:27.126614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c1493cf0-6df3-45ba-8903-5d3f7d707141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e76bc803d784442936b25a7ad36d564', 'request_id': 'req-c1493cf0-6df3-45ba-8903-5d3f7d707141'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:27.133795 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c1493cf0-6df3-45ba-8903-5d3f7d707141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e76bc803d784442936b25a7ad36d564', 'request_id': 'req-c1493cf0-6df3-45ba-8903-5d3f7d707141'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:27.154555 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c1493cf0-6df3-45ba-8903-5d3f7d707141 admin admin] The token cache is being invalidate because project 1e76bc803d784442936b25a7ad36d564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:52:27.154893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c1493cf0-6df3-45ba-8903-5d3f7d707141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:27.155539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c1493cf0-6df3-45ba-8903-5d3f7d707141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:27.157190 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1197/2379] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:52:26 2022] DELETE /identity/v3/projects/1e76bc803d784442936b25a7ad36d564 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:52:27.162227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c395cb1-5076-4ec6-a51b-b730f2be8186 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.162566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c395cb1-5076-4ec6-a51b-b730f2be8186 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.162959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c395cb1-5076-4ec6-a51b-b730f2be8186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.196488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-6c395cb1-5076-4ec6-a51b-b730f2be8186 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:52:27.199155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-6c395cb1-5076-4ec6-a51b-b730f2be8186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:27.269114 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1183/2380] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:52:27.279119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da310b-db97-4a52-a842-a850fec5ffb6 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.279433 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da310b-db97-4a52-a842-a850fec5ffb6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.279836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da310b-db97-4a52-a842-a850fec5ffb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.306995 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-64da310b-db97-4a52-a842-a850fec5ffb6 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:52:27.308819 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-64da310b-db97-4a52-a842-a850fec5ffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:27.323310 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1198/2381] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:52:27.329366 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c395cb1-5076-4ec6-a51b-b730f2be8186 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:27.340786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c395cb1-5076-4ec6-a51b-b730f2be8186 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:27.347655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce1bf6-7081-4ae6-b158-12043f804543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:27.348378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce1bf6-7081-4ae6-b158-12043f804543 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.348686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce1bf6-7081-4ae6-b158-12043f804543 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.348964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce1bf6-7081-4ae6-b158-12043f804543 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.349768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce1bf6-7081-4ae6-b158-12043f804543 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:52:27.350187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce1bf6-7081-4ae6-b158-12043f804543 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:27.351807 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce1bf6-7081-4ae6-b158-12043f804543 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:27.361821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:52:27.367690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1184/2382] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:52:27.368951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef700f-a86b-408c-8948-d75a84560989 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.369497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef700f-a86b-408c-8948-d75a84560989 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.370276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef700f-a86b-408c-8948-d75a84560989 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.370441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce1bf6-7081-4ae6-b158-12043f804543 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:27.371971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1199/2383] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:52:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:52:27.378960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c251f-c6cd-42af-a54a-7a30a5701755 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.378960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c251f-c6cd-42af-a54a-7a30a5701755 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.379273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c251f-c6cd-42af-a54a-7a30a5701755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.383555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce1bf6-7081-4ae6-b158-12043f804543 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:27.389012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015c591-c870-4118-b5d3-f18d269e8b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:27.389681 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015c591-c870-4118-b5d3-f18d269e8b85 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.390121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015c591-c870-4118-b5d3-f18d269e8b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.390545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015c591-c870-4118-b5d3-f18d269e8b85 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.391270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015c591-c870-4118-b5d3-f18d269e8b85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:27.393250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015c591-c870-4118-b5d3-f18d269e8b85 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:27.402996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4c6c251f-c6cd-42af-a54a-7a30a5701755 None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:52:27.404232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4c6c251f-c6cd-42af-a54a-7a30a5701755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:27.435529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1185/2384] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:52:27 2022] POST /identity/v3/projects => generated 399 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:52:27.439540 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015c591-c870-4118-b5d3-f18d269e8b85 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:27.486108 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1200/2385] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:52:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:52:27.499930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c251f-c6cd-42af-a54a-7a30a5701755 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:27.501749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4c6c251f-c6cd-42af-a54a-7a30a5701755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:27.508101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:52:27.523308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015c591-c870-4118-b5d3-f18d269e8b85 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:27.528335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91ff47-fa13-4cc6-ac9e-88a6f739d3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:27.529055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91ff47-fa13-4cc6-ac9e-88a6f739d3a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.529474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91ff47-fa13-4cc6-ac9e-88a6f739d3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.529758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91ff47-fa13-4cc6-ac9e-88a6f739d3a2 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.530485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91ff47-fa13-4cc6-ac9e-88a6f739d3a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:27.532431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91ff47-fa13-4cc6-ac9e-88a6f739d3a2 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:27.561081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1186/2386] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:52:27 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:52:27.565360 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91ff47-fa13-4cc6-ac9e-88a6f739d3a2 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:27.577401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91ff47-fa13-4cc6-ac9e-88a6f739d3a2 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:27.578907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c251f-c6cd-42af-a54a-7a30a5701755 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:27.582928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9499d-aa6b-4e35-b8cc-41b46a0a232b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:27.583903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9499d-aa6b-4e35-b8cc-41b46a0a232b admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.584373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9499d-aa6b-4e35-b8cc-41b46a0a232b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.584859 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9499d-aa6b-4e35-b8cc-41b46a0a232b admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.585712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9499d-aa6b-4e35-b8cc-41b46a0a232b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:27.587469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9499d-aa6b-4e35-b8cc-41b46a0a232b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:27.598843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1187/2387] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:52:27.602650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9499d-aa6b-4e35-b8cc-41b46a0a232b admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:27.614564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9499d-aa6b-4e35-b8cc-41b46a0a232b admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:27.620714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635b916-ce67-4629-995b-1b6d2fd44c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:27.621516 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635b916-ce67-4629-995b-1b6d2fd44c20 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.621888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635b916-ce67-4629-995b-1b6d2fd44c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.622337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635b916-ce67-4629-995b-1b6d2fd44c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.623150 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635b916-ce67-4629-995b-1b6d2fd44c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:27.625437 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635b916-ce67-4629-995b-1b6d2fd44c20 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:27.635384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd29276-9901-4166-b2f7-f9c2a80a5d06 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '925bf2da4df74e86aa84ec62a267d088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:27.636360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd29276-9901-4166-b2f7-f9c2a80a5d06 service glance] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.636857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd29276-9901-4166-b2f7-f9c2a80a5d06 service glance] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.637134 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd29276-9901-4166-b2f7-f9c2a80a5d06 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.638690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1188/2388] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:52:27.642709 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635b916-ce67-4629-995b-1b6d2fd44c20 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:27.653032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635b916-ce67-4629-995b-1b6d2fd44c20 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:27.659704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caae690-eea7-44e0-b1f4-e34c527f5cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:27.660629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caae690-eea7-44e0-b1f4-e34c527f5cef admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.660914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caae690-eea7-44e0-b1f4-e34c527f5cef admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.661188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caae690-eea7-44e0-b1f4-e34c527f5cef admin admin] PATH_INFO: `/v3/projects/78f8d1946e5a48c0b963a66daaba92a2/users/81cce377cefa4938a8c86da7d8685ae1/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.678402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caae690-eea7-44e0-b1f4-e34c527f5cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f8d1946e5a48c0b963a66daaba92a2, user_id=81cce377cefa4938a8c86da7d8685ae1, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:27.682119 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caae690-eea7-44e0-b1f4-e34c527f5cef admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:27.698694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1189/2389] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:52:27 2022] PUT /identity/v3/projects/78f8d1946e5a48c0b963a66daaba92a2/users/81cce377cefa4938a8c86da7d8685ae1/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:52:27.710855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223d58b-c49c-483a-8293-3efee9fa9fcf None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.711322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223d58b-c49c-483a-8293-3efee9fa9fcf None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.711785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223d58b-c49c-483a-8293-3efee9fa9fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.720382 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd29276-9901-4166-b2f7-f9c2a80a5d06 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:27.722907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd29276-9901-4166-b2f7-f9c2a80a5d06 service glance] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:27.741585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1201/2390] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:52:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:52:27.752295 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-8223d58b-c49c-483a-8293-3efee9fa9fcf None None] MFA Rules not processed for user `81cce377cefa4938a8c86da7d8685ae1`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:52:27.753540 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8223d58b-c49c-483a-8293-3efee9fa9fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:27.819019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621ce9e-e663-4f6f-a8f2-60aefeaaa983 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.819360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621ce9e-e663-4f6f-a8f2-60aefeaaa983 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.819749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621ce9e-e663-4f6f-a8f2-60aefeaaa983 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.821312 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1202/2391] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 08:52:27.824628 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1190/2392] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:52:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:52:27.827148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ee81b-eef8-4059-91ad-a8b62ac1f865 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.827709 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ee81b-eef8-4059-91ad-a8b62ac1f865 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.828070 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ee81b-eef8-4059-91ad-a8b62ac1f865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.837944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223d58b-c49c-483a-8293-3efee9fa9fcf None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:27.839398 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8223d58b-c49c-483a-8293-3efee9fa9fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:27.850046 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-610ee81b-eef8-4059-91ad-a8b62ac1f865 None None] MFA Rules not processed for user `62688bc3d8a449c5bc9e0d64cd2462b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:52:27.851341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-610ee81b-eef8-4059-91ad-a8b62ac1f865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:27.853237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:52:27.905732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223d58b-c49c-483a-8293-3efee9fa9fcf None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:27.909224 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1203/2393] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:52:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:52:27.911064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6f2fb-00a9-4bcd-b98c-4dbc1c085319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:27.911764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6f2fb-00a9-4bcd-b98c-4dbc1c085319 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.912052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6f2fb-00a9-4bcd-b98c-4dbc1c085319 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.912332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6f2fb-00a9-4bcd-b98c-4dbc1c085319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.921688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ee81b-eef8-4059-91ad-a8b62ac1f865 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:27.922761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6f2fb-00a9-4bcd-b98c-4dbc1c085319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:27.923117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-610ee81b-eef8-4059-91ad-a8b62ac1f865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:27.924250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6f2fb-00a9-4bcd-b98c-4dbc1c085319 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:27.934347 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1191/2394] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:52:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:52:27.934825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ee81b-eef8-4059-91ad-a8b62ac1f865 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:27.938619 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3816a79-5845-433e-a5fb-836d2f3c515e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:27.939278 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3816a79-5845-433e-a5fb-836d2f3c515e service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:27.939565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3816a79-5845-433e-a5fb-836d2f3c515e service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:27.939844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3816a79-5845-433e-a5fb-836d2f3c515e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:27.945344 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:52:27.948997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3816a79-5845-433e-a5fb-836d2f3c515e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:27.950897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3816a79-5845-433e-a5fb-836d2f3c515e service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:27.959660 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1204/2395] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:52:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:52:28.044351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f225bc-46a0-4ce1-b9a0-ab989de8702a None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:28.044669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f225bc-46a0-4ce1-b9a0-ab989de8702a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:28.044949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f225bc-46a0-4ce1-b9a0-ab989de8702a None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:28.046405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1192/2396] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:52:28.051857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370f15e-7a8f-4730-98a2-656280e8a784 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:28.052371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370f15e-7a8f-4730-98a2-656280e8a784 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:28.052679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370f15e-7a8f-4730-98a2-656280e8a784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:28.073589 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d370f15e-7a8f-4730-98a2-656280e8a784 None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:52:28.074875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d370f15e-7a8f-4730-98a2-656280e8a784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:28.146154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1205/2397] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:52:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:52:28.157322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f225bc-46a0-4ce1-b9a0-ab989de8702a None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:28.159096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-55f225bc-46a0-4ce1-b9a0-ab989de8702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:28.173038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f225bc-46a0-4ce1-b9a0-ab989de8702a None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:28.176672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7bf3b-803b-4474-87d8-6a9139bd1c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:28.177329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7bf3b-803b-4474-87d8-6a9139bd1c53 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:28.177614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7bf3b-803b-4474-87d8-6a9139bd1c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:28.177892 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7bf3b-803b-4474-87d8-6a9139bd1c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:28.189569 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7bf3b-803b-4474-87d8-6a9139bd1c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:28.191010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7bf3b-803b-4474-87d8-6a9139bd1c53 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:28.202480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1193/2398] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:52:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:52:34.840366 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92deae4-43a1-4176-bc7b-7c52bf496fa6 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:34.840946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92deae4-43a1-4176-bc7b-7c52bf496fa6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:34.841053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92deae4-43a1-4176-bc7b-7c52bf496fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:34.859841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d92deae4-43a1-4176-bc7b-7c52bf496fa6 None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:52:34.861066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d92deae4-43a1-4176-bc7b-7c52bf496fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:34.915128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1206/2399] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:52:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:52:34.926926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7bf3b-803b-4474-87d8-6a9139bd1c53 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:34.928379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f7c7bf3b-803b-4474-87d8-6a9139bd1c53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:34.997434 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7bf3b-803b-4474-87d8-6a9139bd1c53 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:35.002766 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b1a4a-41c5-499c-a332-960cfce79e86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:35.003412 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b1a4a-41c5-499c-a332-960cfce79e86 service swift] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:35.003696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b1a4a-41c5-499c-a332-960cfce79e86 service swift] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:35.003976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b1a4a-41c5-499c-a332-960cfce79e86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:35.012531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b1a4a-41c5-499c-a332-960cfce79e86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:35.014134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b1a4a-41c5-499c-a332-960cfce79e86 service swift] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:35.022116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1194/2400] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:52:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:52:36.654914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92deae4-43a1-4176-bc7b-7c52bf496fa6 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:36.656400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d92deae4-43a1-4176-bc7b-7c52bf496fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:36.671824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92deae4-43a1-4176-bc7b-7c52bf496fa6 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:36.677289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44bfaa-d7df-4102-b9ac-ab7e50d332c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:36.677926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44bfaa-d7df-4102-b9ac-ab7e50d332c9 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:36.678209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44bfaa-d7df-4102-b9ac-ab7e50d332c9 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:36.678481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44bfaa-d7df-4102-b9ac-ab7e50d332c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:36.680171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ee44bfaa-d7df-4102-b9ac-ab7e50d332c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:36.737836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44bfaa-d7df-4102-b9ac-ab7e50d332c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:36.739415 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44bfaa-d7df-4102-b9ac-ab7e50d332c9 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:36.748973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1207/2401] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:52:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:52:38.217824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b1a4a-41c5-499c-a332-960cfce79e86 service swift] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:38.290999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b1a4a-41c5-499c-a332-960cfce79e86 service swift] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:38.295475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:38.296350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:38.296655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:38.296927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] PATH_INFO: `/v3/users/81cce377cefa4938a8c86da7d8685ae1` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:38.298163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81cce377cefa4938a8c86da7d8685ae1)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:38.300080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:38.333574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:52:38.345765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81cce377cefa4938a8c86da7d8685ae1'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:38.352470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81cce377cefa4938a8c86da7d8685ae1'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:38.364590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81cce377cefa4938a8c86da7d8685ae1'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:38.371947 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81cce377cefa4938a8c86da7d8685ae1'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:38.372592 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81cce377cefa4938a8c86da7d8685ae1'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:38.378689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1195/2402] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:52:38 2022] DELETE /identity/v3/users/81cce377cefa4938a8c86da7d8685ae1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:52:38.383499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1ea85-7469-434a-99c6-7f512124e986 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:38.383958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1ea85-7469-434a-99c6-7f512124e986 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:38.384200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1ea85-7469-434a-99c6-7f512124e986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:38.402559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-1bf1ea85-7469-434a-99c6-7f512124e986 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:52:38.403765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1bf1ea85-7469-434a-99c6-7f512124e986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:38.429450 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:52:38.463752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1208/2403] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:52:38.473565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:38.474950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:38.533802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b605577-00a6-471c-ad32-6de7772698d0 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:38.537580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290a379-d8b6-4870-a37e-e92203a919d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:38.538270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290a379-d8b6-4870-a37e-e92203a919d9 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:38.538554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290a379-d8b6-4870-a37e-e92203a919d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:38.538837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290a379-d8b6-4870-a37e-e92203a919d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:38.549012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290a379-d8b6-4870-a37e-e92203a919d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:38.550373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290a379-d8b6-4870-a37e-e92203a919d9 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:38.561013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1196/2404] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:52:38.895727 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1ea85-7469-434a-99c6-7f512124e986 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:38.897422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1bf1ea85-7469-434a-99c6-7f512124e986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:38.910089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1ea85-7469-434a-99c6-7f512124e986 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:38.914959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:38.916440 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:38.916766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:38.917037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a admin admin] PATH_INFO: `/v3/projects/78f8d1946e5a48c0b963a66daaba92a2` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:38.918235 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f8d1946e5a48c0b963a66daaba92a2)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:38.920145 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:38.968103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f8d1946e5a48c0b963a66daaba92a2', 'request_id': 'req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:38.977021 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f8d1946e5a48c0b963a66daaba92a2', 'request_id': 'req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:38.983253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f8d1946e5a48c0b963a66daaba92a2', 'request_id': 'req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:38.994212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a admin admin] The token cache is being invalidate because project 78f8d1946e5a48c0b963a66daaba92a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:52:38.994503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:38.995168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f0917efa-68d3-4cc1-9858-2f4a80a6f17a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:52:38.996750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1209/2405] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:52:38 2022] DELETE /identity/v3/projects/78f8d1946e5a48c0b963a66daaba92a2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:52:48.556820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=43,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:52:48.991199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:52:58.746844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290a379-d8b6-4870-a37e-e92203a919d9 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:52:58.748269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1290a379-d8b6-4870-a37e-e92203a919d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:58.830380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290a379-d8b6-4870-a37e-e92203a919d9 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:52:58.903433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fefe8a-aca3-4ed8-aadd-96493b48c5da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:52:58.904135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fefe8a-aca3-4ed8-aadd-96493b48c5da service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:52:58.904422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fefe8a-aca3-4ed8-aadd-96493b48c5da service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:52:58.904757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fefe8a-aca3-4ed8-aadd-96493b48c5da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:52:58.906967 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-77fefe8a-aca3-4ed8-aadd-96493b48c5da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:52:58.968362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fefe8a-aca3-4ed8-aadd-96493b48c5da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:52:58.970161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fefe8a-aca3-4ed8-aadd-96493b48c5da service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:52:58.981277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1197/2406] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:52:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:53:08.977130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=84 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:53:10.881445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9d455-0a90-4b50-a297-c2269fe6bc93 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:53:10.882031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9d455-0a90-4b50-a297-c2269fe6bc93 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:53:10.882151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9d455-0a90-4b50-a297-c2269fe6bc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:53:10.904703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-9ce9d455-0a90-4b50-a297-c2269fe6bc93 None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:53:10.905928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9ce9d455-0a90-4b50-a297-c2269fe6bc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:53:10.971308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1210/2407] 10.222.0.52 () {58 vars in 967 bytes} [Mon Jul 25 08:53:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:53:10.999520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fefe8a-aca3-4ed8-aadd-96493b48c5da service nova] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:53:11.001130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-77fefe8a-aca3-4ed8-aadd-96493b48c5da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:53:11.088287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fefe8a-aca3-4ed8-aadd-96493b48c5da service nova] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:53:11.094622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a69ae-4354-44bd-bac4-40e788439d1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:53:11.095272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a69ae-4354-44bd-bac4-40e788439d1c service swift] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:53:11.095618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a69ae-4354-44bd-bac4-40e788439d1c service swift] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:53:11.095891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a69ae-4354-44bd-bac4-40e788439d1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:53:11.105125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a69ae-4354-44bd-bac4-40e788439d1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:53:11.106701 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a69ae-4354-44bd-bac4-40e788439d1c service swift] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:53:11.117567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1198/2408] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:53:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:53:20.963742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=25 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:53:21.113654 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=32 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:53:21.963298 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9d455-0a90-4b50-a297-c2269fe6bc93 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:53:21.964826 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-9ce9d455-0a90-4b50-a297-c2269fe6bc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:53:21.980777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9d455-0a90-4b50-a297-c2269fe6bc93 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:53:21.986412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b2818-4546-4c98-829b-a20ac718a781 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:53:21.987554 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b2818-4546-4c98-829b-a20ac718a781 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:53:21.987839 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b2818-4546-4c98-829b-a20ac718a781 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:53:21.988122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b2818-4546-4c98-829b-a20ac718a781 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:53:21.989990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-375b2818-4546-4c98-829b-a20ac718a781 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:53:22.056892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b2818-4546-4c98-829b-a20ac718a781 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:53:22.058511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b2818-4546-4c98-829b-a20ac718a781 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:53:22.068900 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1211/2409] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:53:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:53:26.431479 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3343b-d715-45f5-810c-6bdf42ea0a20 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:53:26.432031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3343b-d715-45f5-810c-6bdf42ea0a20 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:53:26.432031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3343b-d715-45f5-810c-6bdf42ea0a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:53:26.453655 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-35c3343b-d715-45f5-810c-6bdf42ea0a20 None None] MFA Rules not processed for user `238f78244df546e5a6fc1a4cd1b73288`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:53:26.455062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-35c3343b-d715-45f5-810c-6bdf42ea0a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:53:26.462233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1199/2410] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:53:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:53:32.063633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=28 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:53:36.448548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:53:38.366481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa577a1b-1af1-4e6b-914a-d60d70903d6c None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:53:38.367042 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa577a1b-1af1-4e6b-914a-d60d70903d6c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:53:38.367042 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa577a1b-1af1-4e6b-914a-d60d70903d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:53:38.387712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-fa577a1b-1af1-4e6b-914a-d60d70903d6c None None] MFA Rules not processed for user `8e682232b7374450a7265e092602a5a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:53:38.388971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fa577a1b-1af1-4e6b-914a-d60d70903d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:53:38.396805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1212/2411] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:53:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:53:38.705026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3343b-d715-45f5-810c-6bdf42ea0a20 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:53:38.706453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-35c3343b-d715-45f5-810c-6bdf42ea0a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:53:38.762347 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3343b-d715-45f5-810c-6bdf42ea0a20 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:53:38.765862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b10c6-1eaf-4fc4-a9f4-657695546818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:53:38.766504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b10c6-1eaf-4fc4-a9f4-657695546818 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:53:38.766782 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b10c6-1eaf-4fc4-a9f4-657695546818 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:53:38.767052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b10c6-1eaf-4fc4-a9f4-657695546818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:53:38.768822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-317b10c6-1eaf-4fc4-a9f4-657695546818 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:53:38.778219 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b10c6-1eaf-4fc4-a9f4-657695546818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:53:38.779630 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b10c6-1eaf-4fc4-a9f4-657695546818 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:53:38.787566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1200/2412] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:53:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:53:48.382448 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:53:48.783956 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=26 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:54:29.130366 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa577a1b-1af1-4e6b-914a-d60d70903d6c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:54:29.132345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fa577a1b-1af1-4e6b-914a-d60d70903d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:29.243836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa577a1b-1af1-4e6b-914a-d60d70903d6c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:54:29.249955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44830f0b-efe0-4430-a9cb-a043ee8140b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:54:29.250617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44830f0b-efe0-4430-a9cb-a043ee8140b4 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:29.250903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44830f0b-efe0-4430-a9cb-a043ee8140b4 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:29.252045 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44830f0b-efe0-4430-a9cb-a043ee8140b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:29.253392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-44830f0b-efe0-4430-a9cb-a043ee8140b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:29.322469 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44830f0b-efe0-4430-a9cb-a043ee8140b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:54:29.324073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44830f0b-efe0-4430-a9cb-a043ee8140b4 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:54:29.333397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1213/2413] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:54:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:54:35.401319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b10c6-1eaf-4fc4-a9f4-657695546818 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:54:35.443643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b10c6-1eaf-4fc4-a9f4-657695546818 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:54:35.452227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72252bd3-f045-44b8-9c09-af09d8f4b558 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:54:35.453277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72252bd3-f045-44b8-9c09-af09d8f4b558 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:35.453657 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72252bd3-f045-44b8-9c09-af09d8f4b558 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:35.454057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72252bd3-f045-44b8-9c09-af09d8f4b558 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:35.457325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-72252bd3-f045-44b8-9c09-af09d8f4b558 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:35.557483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72252bd3-f045-44b8-9c09-af09d8f4b558 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:54:35.560037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72252bd3-f045-44b8-9c09-af09d8f4b558 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:54:35.570744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1201/2414] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:54:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:54:36.109986 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44830f0b-efe0-4430-a9cb-a043ee8140b4 service nova] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:54:36.113014 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-44830f0b-efe0-4430-a9cb-a043ee8140b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:36.374966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44830f0b-efe0-4430-a9cb-a043ee8140b4 service nova] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:54:36.396791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95f4db-16a6-4519-b5e8-cd553c7ca5e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:54:36.398419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95f4db-16a6-4519-b5e8-cd553c7ca5e9 service placement] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:36.398888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95f4db-16a6-4519-b5e8-cd553c7ca5e9 service placement] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:36.399481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95f4db-16a6-4519-b5e8-cd553c7ca5e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:36.404210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8b95f4db-16a6-4519-b5e8-cd553c7ca5e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:36.415390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:54:36.428850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95f4db-16a6-4519-b5e8-cd553c7ca5e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:54:36.431035 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95f4db-16a6-4519-b5e8-cd553c7ca5e9 service placement] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:54:36.448536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1214/2415] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:54:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:54:36.613397 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72252bd3-f045-44b8-9c09-af09d8f4b558 service nova] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:54:36.636376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72252bd3-f045-44b8-9c09-af09d8f4b558 service nova] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:54:36.647296 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91d630-f015-4ade-aab0-be058a021385 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:54:36.649210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91d630-f015-4ade-aab0-be058a021385 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:36.649932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91d630-f015-4ade-aab0-be058a021385 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:36.650387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91d630-f015-4ade-aab0-be058a021385 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:36.652769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8e91d630-f015-4ade-aab0-be058a021385 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:36.677124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91d630-f015-4ade-aab0-be058a021385 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:54:36.684520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91d630-f015-4ade-aab0-be058a021385 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:54:36.721690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1202/2416] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:54:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:54:40.691209 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868528e-48a8-4350-8ec0-049beaffd854 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:40.691732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868528e-48a8-4350-8ec0-049beaffd854 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:40.692288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868528e-48a8-4350-8ec0-049beaffd854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:40.723313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7868528e-48a8-4350-8ec0-049beaffd854 None None] MFA Rules not processed for user `718ca333456f4de5b98f191ab40cca0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:54:40.725491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7868528e-48a8-4350-8ec0-049beaffd854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:40.810388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1215/2417] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:54:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:54:41.899563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ea2ea-875a-469d-a1bc-b44aab2e1ab9 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:41.900078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ea2ea-875a-469d-a1bc-b44aab2e1ab9 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:41.900188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ea2ea-875a-469d-a1bc-b44aab2e1ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:41.923351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-d73ea2ea-875a-469d-a1bc-b44aab2e1ab9 None None] MFA Rules not processed for user `d3f0b30685e94f83883d682731644a89`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:54:41.924629 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d73ea2ea-875a-469d-a1bc-b44aab2e1ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:41.932911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1203/2418] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:54:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:54:46.987083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba2ee-dc65-4c32-b327-f931a2f8ba9a None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:46.987693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba2ee-dc65-4c32-b327-f931a2f8ba9a None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:46.992707 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba2ee-dc65-4c32-b327-f931a2f8ba9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:47.053039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c9eba2ee-dc65-4c32-b327-f931a2f8ba9a None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:54:47.066079 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c9eba2ee-dc65-4c32-b327-f931a2f8ba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:47.075364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1216/2419] 10.222.0.52 () {58 vars in 967 bytes} [Mon Jul 25 08:54:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:54:47.184721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ea2ea-875a-469d-a1bc-b44aab2e1ab9 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:54:47.217411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ea2ea-875a-469d-a1bc-b44aab2e1ab9 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:54:47.224679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0b681-afff-420e-8d40-aa080fae3ea7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:54:47.225854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0b681-afff-420e-8d40-aa080fae3ea7 service swift] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:47.226381 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0b681-afff-420e-8d40-aa080fae3ea7 service swift] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:47.226661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0b681-afff-420e-8d40-aa080fae3ea7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:47.243756 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0b681-afff-420e-8d40-aa080fae3ea7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:54:47.245662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0b681-afff-420e-8d40-aa080fae3ea7 service swift] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:54:47.263193 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1204/2420] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:54:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:54:47.825102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eba2ee-dc65-4c32-b327-f931a2f8ba9a None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:54:47.828191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c9eba2ee-dc65-4c32-b327-f931a2f8ba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:47.907453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eba2ee-dc65-4c32-b327-f931a2f8ba9a None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:54:47.914850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c559482-a23c-4b23-8914-ea2bfbdfb7bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:54:47.916018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c559482-a23c-4b23-8914-ea2bfbdfb7bd service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:47.916503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c559482-a23c-4b23-8914-ea2bfbdfb7bd service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:47.917287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c559482-a23c-4b23-8914-ea2bfbdfb7bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:47.922811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3c559482-a23c-4b23-8914-ea2bfbdfb7bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:48.043038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c559482-a23c-4b23-8914-ea2bfbdfb7bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:54:48.044536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c559482-a23c-4b23-8914-ea2bfbdfb7bd service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:54:48.073043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1217/2421] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:54:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:54:51.908636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0b681-afff-420e-8d40-aa080fae3ea7 service swift] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:54:51.922136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0b681-afff-420e-8d40-aa080fae3ea7 service swift] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:54:51.932281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b02ed-3a3c-4032-90e3-b6299c147b8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:54:51.933833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b02ed-3a3c-4032-90e3-b6299c147b8d service placement] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:51.934428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b02ed-3a3c-4032-90e3-b6299c147b8d service placement] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:51.934878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b02ed-3a3c-4032-90e3-b6299c147b8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:51.937324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c31b02ed-3a3c-4032-90e3-b6299c147b8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:51.956552 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b02ed-3a3c-4032-90e3-b6299c147b8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:54:51.959965 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b02ed-3a3c-4032-90e3-b6299c147b8d service placement] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:54:51.982771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1205/2422] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:54:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:54:56.480153 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c559482-a23c-4b23-8914-ea2bfbdfb7bd service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:54:56.481825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3c559482-a23c-4b23-8914-ea2bfbdfb7bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:56.504117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c559482-a23c-4b23-8914-ea2bfbdfb7bd service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:54:56.511232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:54:56.512472 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:56.512853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:56.513380 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] PATH_INFO: `/v3/users/cd036c73e5c84fa1ba548be4eae80804` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:56.514742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd036c73e5c84fa1ba548be4eae80804)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:54:56.518270 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:54:56.546607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:54:56.585158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd036c73e5c84fa1ba548be4eae80804'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:56.587051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd036c73e5c84fa1ba548be4eae80804'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:56.631622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd036c73e5c84fa1ba548be4eae80804'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:56.645454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd036c73e5c84fa1ba548be4eae80804'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:56.652463 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd036c73e5c84fa1ba548be4eae80804'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:56.659731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1218/2423] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:54:56 2022] DELETE /identity/v3/users/cd036c73e5c84fa1ba548be4eae80804 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:54:56.875161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b02ed-3a3c-4032-90e3-b6299c147b8d service placement] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:54:56.887586 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b02ed-3a3c-4032-90e3-b6299c147b8d service placement] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:54:56.972248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:54:56.973170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:56.973460 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:56.973729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] PATH_INFO: `/v3/projects/3404b1a99987401183c8e3e851b382db` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:56.974929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3404b1a99987401183c8e3e851b382db)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:54:56.976870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:54:57.021940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:54:57.026075 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3404b1a99987401183c8e3e851b382db', 'request_id': 'req-ff506400-b565-470c-97f6-8e9da65b2795'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.034325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3404b1a99987401183c8e3e851b382db', 'request_id': 'req-ff506400-b565-470c-97f6-8e9da65b2795'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.040467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3404b1a99987401183c8e3e851b382db', 'request_id': 'req-ff506400-b565-470c-97f6-8e9da65b2795'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.052145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] The token cache is being invalidate because project 3404b1a99987401183c8e3e851b382db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:54:57.052425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.053118 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.054717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1206/2424] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:54:56 2022] DELETE /identity/v3/projects/3404b1a99987401183c8e3e851b382db => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:54:57.058897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:54:57.060352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:57.075157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc18699-46a8-49c7-a3a9-7e17a4e65f3a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:54:57.142107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:54:57.143398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:57.143825 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:57.144297 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] PATH_INFO: `/v3/users/0aa05c58ebca42b28d3a15ec222aee76` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:57.146854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa05c58ebca42b28d3a15ec222aee76)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:54:57.149888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:54:57.208053 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa05c58ebca42b28d3a15ec222aee76'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.208891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa05c58ebca42b28d3a15ec222aee76'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.223844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa05c58ebca42b28d3a15ec222aee76'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.238785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa05c58ebca42b28d3a15ec222aee76'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.252152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa05c58ebca42b28d3a15ec222aee76'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.262850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1219/2425] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:54:57 2022] DELETE /identity/v3/users/0aa05c58ebca42b28d3a15ec222aee76 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:54:57.459365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:54:57.461510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:54:57.480614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff506400-b565-470c-97f6-8e9da65b2795 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:54:57.545194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:54:57.546074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:54:57.546357 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:54:57.546663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] PATH_INFO: `/v3/projects/a4c21500aa804b7793d171a941d61706` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:54:57.547977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4c21500aa804b7793d171a941d61706)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:54:57.550420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:54:57.600283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4c21500aa804b7793d171a941d61706', 'request_id': 'req-328e9e6f-d880-43b9-80dd-5c7612de941d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.609593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4c21500aa804b7793d171a941d61706', 'request_id': 'req-328e9e6f-d880-43b9-80dd-5c7612de941d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.616805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4c21500aa804b7793d171a941d61706', 'request_id': 'req-328e9e6f-d880-43b9-80dd-5c7612de941d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.628175 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] The token cache is being invalidate because project a4c21500aa804b7793d171a941d61706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:54:57.628470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.629270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:54:57.630830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1207/2426] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:54:57 2022] DELETE /identity/v3/projects/a4c21500aa804b7793d171a941d61706 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:07.256063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=73,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:07.625761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=59,keystone:DELETE=11,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:11.572309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:11.573990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:11.643503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd26fc9-0fc0-48d5-9d1c-57dc57dd08d8 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:11.696824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822abf2-7fc8-458c-9f49-326b1cc16da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:11.697493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822abf2-7fc8-458c-9f49-326b1cc16da5 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:11.697782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822abf2-7fc8-458c-9f49-326b1cc16da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:11.698062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822abf2-7fc8-458c-9f49-326b1cc16da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:11.700008 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c822abf2-7fc8-458c-9f49-326b1cc16da5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:11.789712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822abf2-7fc8-458c-9f49-326b1cc16da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:11.791076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822abf2-7fc8-458c-9f49-326b1cc16da5 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:11.803007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1220/2427] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:55:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:55:15.315125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:15.319138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:15.340497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e9e6f-d880-43b9-80dd-5c7612de941d admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:15.349705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26581929-d1f9-4f1c-bda8-a44304a63678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:15.351260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26581929-d1f9-4f1c-bda8-a44304a63678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:15.351753 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26581929-d1f9-4f1c-bda8-a44304a63678 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:15.352253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26581929-d1f9-4f1c-bda8-a44304a63678 admin admin] PATH_INFO: `/v3/users/238f78244df546e5a6fc1a4cd1b73288` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:15.354689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26581929-d1f9-4f1c-bda8-a44304a63678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=238f78244df546e5a6fc1a4cd1b73288)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:15.358713 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26581929-d1f9-4f1c-bda8-a44304a63678 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:15.433025 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-26581929-d1f9-4f1c-bda8-a44304a63678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '238f78244df546e5a6fc1a4cd1b73288'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:15.449202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-26581929-d1f9-4f1c-bda8-a44304a63678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '238f78244df546e5a6fc1a4cd1b73288'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:15.468916 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-26581929-d1f9-4f1c-bda8-a44304a63678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '238f78244df546e5a6fc1a4cd1b73288'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:15.482618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-26581929-d1f9-4f1c-bda8-a44304a63678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '238f78244df546e5a6fc1a4cd1b73288'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:15.484049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-26581929-d1f9-4f1c-bda8-a44304a63678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '238f78244df546e5a6fc1a4cd1b73288'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:15.496026 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1208/2428] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:55:15 2022] DELETE /identity/v3/users/238f78244df546e5a6fc1a4cd1b73288 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:15.724713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822abf2-7fc8-458c-9f49-326b1cc16da5 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:15.738061 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822abf2-7fc8-458c-9f49-326b1cc16da5 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:15.803591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4299c2-41c2-428e-9501-4bb4db14889a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:15.804390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4299c2-41c2-428e-9501-4bb4db14889a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:15.804708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4299c2-41c2-428e-9501-4bb4db14889a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:15.804989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4299c2-41c2-428e-9501-4bb4db14889a admin admin] PATH_INFO: `/v3/projects/7e61cbb537384f549eee9ec4c92d9d77` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:15.806262 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4299c2-41c2-428e-9501-4bb4db14889a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e61cbb537384f549eee9ec4c92d9d77)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:15.808213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4299c2-41c2-428e-9501-4bb4db14889a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:15.817557 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:15.861285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2f4299c2-41c2-428e-9501-4bb4db14889a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e61cbb537384f549eee9ec4c92d9d77', 'request_id': 'req-2f4299c2-41c2-428e-9501-4bb4db14889a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:15.869794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2f4299c2-41c2-428e-9501-4bb4db14889a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e61cbb537384f549eee9ec4c92d9d77', 'request_id': 'req-2f4299c2-41c2-428e-9501-4bb4db14889a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:15.876404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2f4299c2-41c2-428e-9501-4bb4db14889a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e61cbb537384f549eee9ec4c92d9d77', 'request_id': 'req-2f4299c2-41c2-428e-9501-4bb4db14889a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:15.888339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2f4299c2-41c2-428e-9501-4bb4db14889a admin admin] The token cache is being invalidate because project 7e61cbb537384f549eee9ec4c92d9d77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:55:15.888664 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2f4299c2-41c2-428e-9501-4bb4db14889a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:15.889462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2f4299c2-41c2-428e-9501-4bb4db14889a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:15.891791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1221/2429] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:55:15 2022] DELETE /identity/v3/projects/7e61cbb537384f549eee9ec4c92d9d77 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:15.896980 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a262475-b6ac-4e2a-a219-592c2594b790 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:15.897276 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a262475-b6ac-4e2a-a219-592c2594b790 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:15.897575 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a262475-b6ac-4e2a-a219-592c2594b790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:15.963270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-7a262475-b6ac-4e2a-a219-592c2594b790 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:15.964504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7a262475-b6ac-4e2a-a219-592c2594b790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:16.032413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1209/2430] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:16.040284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a54043-cbd7-4034-863a-e53fbf3b662b None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:16.040585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a54043-cbd7-4034-863a-e53fbf3b662b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:16.040888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a54043-cbd7-4034-863a-e53fbf3b662b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:16.061656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-47a54043-cbd7-4034-863a-e53fbf3b662b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:16.062905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-47a54043-cbd7-4034-863a-e53fbf3b662b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:16.071274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1222/2431] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:16.075696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a262475-b6ac-4e2a-a219-592c2594b790 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:16.086968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a262475-b6ac-4e2a-a219-592c2594b790 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:16.092751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d595a-70ad-4bcd-8aeb-4c2467711e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:16.093527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d595a-70ad-4bcd-8aeb-4c2467711e09 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:16.093812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d595a-70ad-4bcd-8aeb-4c2467711e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:16.094086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d595a-70ad-4bcd-8aeb-4c2467711e09 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:16.095488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d595a-70ad-4bcd-8aeb-4c2467711e09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:55:16.095794 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d595a-70ad-4bcd-8aeb-4c2467711e09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:16.099050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d595a-70ad-4bcd-8aeb-4c2467711e09 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:16.111006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1210/2432] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:55:16.115667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a54043-cbd7-4034-863a-e53fbf3b662b None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:16.126184 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a54043-cbd7-4034-863a-e53fbf3b662b None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:16.131417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c418f5-31e7-446a-8a75-68ffba878a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:16.132083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c418f5-31e7-446a-8a75-68ffba878a4d admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:16.132367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c418f5-31e7-446a-8a75-68ffba878a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:16.132681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c418f5-31e7-446a-8a75-68ffba878a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:16.133382 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c418f5-31e7-446a-8a75-68ffba878a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:16.135420 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c418f5-31e7-446a-8a75-68ffba878a4d admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:16.165799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1223/2433] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:55:16 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:55:16.171439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d595a-70ad-4bcd-8aeb-4c2467711e09 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:16.182289 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d595a-70ad-4bcd-8aeb-4c2467711e09 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:16.210861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:16.246293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0a7c4-0020-461f-9197-0b3f2f5fcf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:16.246896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0a7c4-0020-461f-9197-0b3f2f5fcf06 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:16.247205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0a7c4-0020-461f-9197-0b3f2f5fcf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:16.247482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0a7c4-0020-461f-9197-0b3f2f5fcf06 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:16.248213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0a7c4-0020-461f-9197-0b3f2f5fcf06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:16.250240 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0a7c4-0020-461f-9197-0b3f2f5fcf06 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:16.286411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1211/2434] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:55:16 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:55:16.291193 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c418f5-31e7-446a-8a75-68ffba878a4d admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:16.302542 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c418f5-31e7-446a-8a75-68ffba878a4d admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:16.307685 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54ce6b-69b6-45af-bb62-9e39fde8b178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:16.308378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54ce6b-69b6-45af-bb62-9e39fde8b178 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:16.308699 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54ce6b-69b6-45af-bb62-9e39fde8b178 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:16.308975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54ce6b-69b6-45af-bb62-9e39fde8b178 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:16.309875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb54ce6b-69b6-45af-bb62-9e39fde8b178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:16.311358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb54ce6b-69b6-45af-bb62-9e39fde8b178 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:16.323160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1224/2435] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:55:16.327817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0a7c4-0020-461f-9197-0b3f2f5fcf06 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:16.339155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0a7c4-0020-461f-9197-0b3f2f5fcf06 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:16.344450 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4438f4-0612-44a3-9800-020fc025f34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:16.345205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4438f4-0612-44a3-9800-020fc025f34b admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:16.345493 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4438f4-0612-44a3-9800-020fc025f34b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:16.345799 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4438f4-0612-44a3-9800-020fc025f34b admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:16.346554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4438f4-0612-44a3-9800-020fc025f34b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:16.348207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4438f4-0612-44a3-9800-020fc025f34b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:16.360530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1212/2436] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:55:16.365083 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54ce6b-69b6-45af-bb62-9e39fde8b178 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:16.376704 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54ce6b-69b6-45af-bb62-9e39fde8b178 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:16.381804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41ed52-7a97-4486-a291-ad6d45084262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:16.382754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41ed52-7a97-4486-a291-ad6d45084262 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:16.383066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41ed52-7a97-4486-a291-ad6d45084262 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:16.383346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41ed52-7a97-4486-a291-ad6d45084262 admin admin] PATH_INFO: `/v3/projects/161188f13b02425987c1a1ac5c497751/users/2f383ccb6ba740eca9543214d828d295/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:16.401752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41ed52-7a97-4486-a291-ad6d45084262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=161188f13b02425987c1a1ac5c497751, user_id=2f383ccb6ba740eca9543214d828d295, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:16.404534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41ed52-7a97-4486-a291-ad6d45084262 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:16.419260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1225/2437] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:55:16 2022] PUT /identity/v3/projects/161188f13b02425987c1a1ac5c497751/users/2f383ccb6ba740eca9543214d828d295/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:16.430262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7decb7f-4b27-4da0-879f-cffba54e7187 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:16.430262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7decb7f-4b27-4da0-879f-cffba54e7187 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:16.430262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7decb7f-4b27-4da0-879f-cffba54e7187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:16.463405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-d7decb7f-4b27-4da0-879f-cffba54e7187 None None] MFA Rules not processed for user `2f383ccb6ba740eca9543214d828d295`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:16.464793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d7decb7f-4b27-4da0-879f-cffba54e7187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:16.537571 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1213/2438] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:16.556198 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41ed52-7a97-4486-a291-ad6d45084262 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:16.558430 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ce41ed52-7a97-4486-a291-ad6d45084262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:16.636915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41ed52-7a97-4486-a291-ad6d45084262 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:16.643027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df30d2-4a0b-4bf8-9dfc-d0c39c376f07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:16.643680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df30d2-4a0b-4bf8-9dfc-d0c39c376f07 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:16.643958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df30d2-4a0b-4bf8-9dfc-d0c39c376f07 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:16.644234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df30d2-4a0b-4bf8-9dfc-d0c39c376f07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:16.653043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df30d2-4a0b-4bf8-9dfc-d0c39c376f07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:16.654802 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df30d2-4a0b-4bf8-9dfc-d0c39c376f07 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:16.664432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1226/2439] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:55:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:55:26.093572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7decb7f-4b27-4da0-879f-cffba54e7187 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:26.171059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7decb7f-4b27-4da0-879f-cffba54e7187 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:26.178003 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:26.179349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:26.179768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:26.180176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] PATH_INFO: `/v3/users/2f383ccb6ba740eca9543214d828d295` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:26.182043 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f383ccb6ba740eca9543214d828d295)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:26.185163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:26.234408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:26.241802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f383ccb6ba740eca9543214d828d295'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:26.253101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f383ccb6ba740eca9543214d828d295'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:26.273932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f383ccb6ba740eca9543214d828d295'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:26.281235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f383ccb6ba740eca9543214d828d295'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:26.281977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f383ccb6ba740eca9543214d828d295'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:26.288038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1214/2440] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:55:26 2022] DELETE /identity/v3/users/2f383ccb6ba740eca9543214d828d295 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:26.293989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fb65c-75ad-43f3-afc0-16a2cf01b286 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:26.294675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fb65c-75ad-43f3-afc0-16a2cf01b286 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:26.294675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fb65c-75ad-43f3-afc0-16a2cf01b286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:26.315499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-bc7fb65c-75ad-43f3-afc0-16a2cf01b286 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:26.316775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-bc7fb65c-75ad-43f3-afc0-16a2cf01b286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:26.330100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:26.386941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1227/2441] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:26.398229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:26.400529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:26.495895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be350957-093f-4fb6-82f6-47af51f00cbc admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:26.499834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7b06e-19e5-490d-b601-c676ef163ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:26.500816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7b06e-19e5-490d-b601-c676ef163ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:26.501105 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7b06e-19e5-490d-b601-c676ef163ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:26.501384 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7b06e-19e5-490d-b601-c676ef163ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:26.511712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7b06e-19e5-490d-b601-c676ef163ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:26.513349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7b06e-19e5-490d-b601-c676ef163ad0 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:26.524599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1215/2442] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:55:26.823025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fb65c-75ad-43f3-afc0-16a2cf01b286 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:26.824652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-bc7fb65c-75ad-43f3-afc0-16a2cf01b286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:26.838058 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fb65c-75ad-43f3-afc0-16a2cf01b286 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:26.846554 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:26.847566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:26.847959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:26.848332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba admin admin] PATH_INFO: `/v3/projects/161188f13b02425987c1a1ac5c497751` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:26.849813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=161188f13b02425987c1a1ac5c497751)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:26.851901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:26.909953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '161188f13b02425987c1a1ac5c497751', 'request_id': 'req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:26.919166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '161188f13b02425987c1a1ac5c497751', 'request_id': 'req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:26.926211 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '161188f13b02425987c1a1ac5c497751', 'request_id': 'req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:26.938418 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba admin admin] The token cache is being invalidate because project 161188f13b02425987c1a1ac5c497751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:55:26.938696 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:26.939412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-3433fb9f-32be-45bb-b3ca-6ce6917c3bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:26.941021 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1228/2443] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:55:26 2022] DELETE /identity/v3/projects/161188f13b02425987c1a1ac5c497751 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:26.946070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813bc25-8386-4523-8b42-2b37481b02b4 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:26.946394 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813bc25-8386-4523-8b42-2b37481b02b4 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:26.946740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813bc25-8386-4523-8b42-2b37481b02b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:26.971030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-c813bc25-8386-4523-8b42-2b37481b02b4 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:26.972271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c813bc25-8386-4523-8b42-2b37481b02b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:27.038281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1216/2444] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:27.046098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f2146-1992-4303-8bbc-238b347dc2ce None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:27.046392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f2146-1992-4303-8bbc-238b347dc2ce None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:27.046679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f2146-1992-4303-8bbc-238b347dc2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:27.071328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-747f2146-1992-4303-8bbc-238b347dc2ce None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:27.072511 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-747f2146-1992-4303-8bbc-238b347dc2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:27.081619 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1229/2445] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:27.085947 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813bc25-8386-4523-8b42-2b37481b02b4 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:27.096991 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813bc25-8386-4523-8b42-2b37481b02b4 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:27.101750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e2157-344f-477f-a07d-daec06e9a173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:27.102423 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e2157-344f-477f-a07d-daec06e9a173 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:27.102708 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e2157-344f-477f-a07d-daec06e9a173 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:27.102984 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e2157-344f-477f-a07d-daec06e9a173 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:27.103726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e2157-344f-477f-a07d-daec06e9a173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:55:27.104032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e2157-344f-477f-a07d-daec06e9a173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:27.105656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e2157-344f-477f-a07d-daec06e9a173 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:27.118482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1217/2446] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:55:27.123541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f2146-1992-4303-8bbc-238b347dc2ce None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:27.133582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f2146-1992-4303-8bbc-238b347dc2ce None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:27.138538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebb5f2-4c4d-4ea7-916a-da68859e4a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:27.139204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebb5f2-4c4d-4ea7-916a-da68859e4a3a admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:27.139485 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebb5f2-4c4d-4ea7-916a-da68859e4a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:27.139762 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebb5f2-4c4d-4ea7-916a-da68859e4a3a admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:27.140975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebb5f2-4c4d-4ea7-916a-da68859e4a3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:27.142985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebb5f2-4c4d-4ea7-916a-da68859e4a3a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:27.171715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1230/2447] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:55:27 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:55:27.176028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e2157-344f-477f-a07d-daec06e9a173 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:27.186317 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e2157-344f-477f-a07d-daec06e9a173 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:27.232159 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:27.246838 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bcfad-727c-4c84-8a4e-58ab4096f86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:27.247581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bcfad-727c-4c84-8a4e-58ab4096f86b admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:27.247862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bcfad-727c-4c84-8a4e-58ab4096f86b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:27.248140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bcfad-727c-4c84-8a4e-58ab4096f86b admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:27.248898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bcfad-727c-4c84-8a4e-58ab4096f86b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:27.250843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bcfad-727c-4c84-8a4e-58ab4096f86b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:27.274700 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1218/2448] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:55:27 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:55:27.279199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebb5f2-4c4d-4ea7-916a-da68859e4a3a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:27.290731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebb5f2-4c4d-4ea7-916a-da68859e4a3a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:27.295506 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30459b74-9caa-46c4-ab5b-01da354ecc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:27.296188 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30459b74-9caa-46c4-ab5b-01da354ecc60 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:27.296479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30459b74-9caa-46c4-ab5b-01da354ecc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:27.296876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30459b74-9caa-46c4-ab5b-01da354ecc60 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:27.297636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30459b74-9caa-46c4-ab5b-01da354ecc60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:27.299275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30459b74-9caa-46c4-ab5b-01da354ecc60 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:27.310268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1231/2449] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:55:27.314789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bcfad-727c-4c84-8a4e-58ab4096f86b admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:27.324982 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bcfad-727c-4c84-8a4e-58ab4096f86b admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:27.330096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba2da0-8e5e-4d3f-bc75-c0b692253c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:27.330761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba2da0-8e5e-4d3f-bc75-c0b692253c81 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:27.331043 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba2da0-8e5e-4d3f-bc75-c0b692253c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:27.331319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba2da0-8e5e-4d3f-bc75-c0b692253c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:27.332051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba2da0-8e5e-4d3f-bc75-c0b692253c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:27.333687 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba2da0-8e5e-4d3f-bc75-c0b692253c81 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:27.344820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1219/2450] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:55:27.349095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30459b74-9caa-46c4-ab5b-01da354ecc60 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:27.359392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30459b74-9caa-46c4-ab5b-01da354ecc60 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:27.364031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec267d-f087-45fb-bac9-6c9224e7b9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:27.364960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec267d-f087-45fb-bac9-6c9224e7b9c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:27.365246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec267d-f087-45fb-bac9-6c9224e7b9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:27.365535 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec267d-f087-45fb-bac9-6c9224e7b9c8 admin admin] PATH_INFO: `/v3/projects/ebd6083e6f18454781ae160d105c568e/users/51cee82858d9446da50aab697dfcc3f3/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:27.380923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec267d-f087-45fb-bac9-6c9224e7b9c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd6083e6f18454781ae160d105c568e, user_id=51cee82858d9446da50aab697dfcc3f3, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:27.383589 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec267d-f087-45fb-bac9-6c9224e7b9c8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:27.398060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1232/2451] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:55:27 2022] PUT /identity/v3/projects/ebd6083e6f18454781ae160d105c568e/users/51cee82858d9446da50aab697dfcc3f3/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:27.410585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce28d8a-6419-425a-8b71-0cc1b8552fde None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:27.410866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce28d8a-6419-425a-8b71-0cc1b8552fde None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:27.411178 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce28d8a-6419-425a-8b71-0cc1b8552fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:27.440622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3ce28d8a-6419-425a-8b71-0cc1b8552fde None None] MFA Rules not processed for user `51cee82858d9446da50aab697dfcc3f3`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:27.441848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3ce28d8a-6419-425a-8b71-0cc1b8552fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:27.541379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1220/2452] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:27.554787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec267d-f087-45fb-bac9-6c9224e7b9c8 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:27.556507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e7ec267d-f087-45fb-bac9-6c9224e7b9c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:27.601548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:27.638734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec267d-f087-45fb-bac9-6c9224e7b9c8 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:27.645258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8b093-e036-4f3a-8fad-b244dc08eb3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:27.645942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8b093-e036-4f3a-8fad-b244dc08eb3d service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:27.646213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8b093-e036-4f3a-8fad-b244dc08eb3d service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:27.646492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8b093-e036-4f3a-8fad-b244dc08eb3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:27.655547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8b093-e036-4f3a-8fad-b244dc08eb3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:27.657257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8b093-e036-4f3a-8fad-b244dc08eb3d service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:27.667246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1233/2453] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:55:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:55:27.895569 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce28d8a-6419-425a-8b71-0cc1b8552fde None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:27.958582 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce28d8a-6419-425a-8b71-0cc1b8552fde None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:27.963142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:27.964029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:27.964312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:27.964608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] PATH_INFO: `/v3/users/51cee82858d9446da50aab697dfcc3f3` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:27.965821 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51cee82858d9446da50aab697dfcc3f3)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:27.967761 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:28.015474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51cee82858d9446da50aab697dfcc3f3'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:28.023062 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51cee82858d9446da50aab697dfcc3f3'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:28.025058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:28.038464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51cee82858d9446da50aab697dfcc3f3'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:28.045431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51cee82858d9446da50aab697dfcc3f3'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:28.046112 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51cee82858d9446da50aab697dfcc3f3'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:28.052497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1221/2454] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:55:27 2022] DELETE /identity/v3/users/51cee82858d9446da50aab697dfcc3f3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:28.057072 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde27f5e-47c6-4005-a041-3d0bc735a34d None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:28.057367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde27f5e-47c6-4005-a041-3d0bc735a34d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:28.057652 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde27f5e-47c6-4005-a041-3d0bc735a34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:28.077267 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-dde27f5e-47c6-4005-a041-3d0bc735a34d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:28.078498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-dde27f5e-47c6-4005-a041-3d0bc735a34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:28.137558 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1234/2455] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:28.147572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:28.149175 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:28.208445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85186011-adc9-4a22-abb2-c218f729c666 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:28.212198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef4ddf-4850-42d2-b39d-7b907312d264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:28.212891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef4ddf-4850-42d2-b39d-7b907312d264 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:28.213176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef4ddf-4850-42d2-b39d-7b907312d264 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:28.213454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef4ddf-4850-42d2-b39d-7b907312d264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:28.222749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef4ddf-4850-42d2-b39d-7b907312d264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:28.224183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef4ddf-4850-42d2-b39d-7b907312d264 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:28.234053 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1222/2456] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:55:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:55:28.513329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde27f5e-47c6-4005-a041-3d0bc735a34d None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:28.514945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-dde27f5e-47c6-4005-a041-3d0bc735a34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:28.527460 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde27f5e-47c6-4005-a041-3d0bc735a34d None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:28.532598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d03b10-399b-4f57-8790-a796d979ea3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:28.533480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d03b10-399b-4f57-8790-a796d979ea3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:28.533766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d03b10-399b-4f57-8790-a796d979ea3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:28.534044 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d03b10-399b-4f57-8790-a796d979ea3d admin admin] PATH_INFO: `/v3/projects/ebd6083e6f18454781ae160d105c568e` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:28.535287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d03b10-399b-4f57-8790-a796d979ea3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd6083e6f18454781ae160d105c568e)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:28.537307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d03b10-399b-4f57-8790-a796d979ea3d admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:28.585878 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d9d03b10-399b-4f57-8790-a796d979ea3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd6083e6f18454781ae160d105c568e', 'request_id': 'req-d9d03b10-399b-4f57-8790-a796d979ea3d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:28.593034 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d9d03b10-399b-4f57-8790-a796d979ea3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd6083e6f18454781ae160d105c568e', 'request_id': 'req-d9d03b10-399b-4f57-8790-a796d979ea3d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:28.599459 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d9d03b10-399b-4f57-8790-a796d979ea3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd6083e6f18454781ae160d105c568e', 'request_id': 'req-d9d03b10-399b-4f57-8790-a796d979ea3d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:28.611875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d9d03b10-399b-4f57-8790-a796d979ea3d admin admin] The token cache is being invalidate because project ebd6083e6f18454781ae160d105c568e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:55:28.612158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d9d03b10-399b-4f57-8790-a796d979ea3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:28.612853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d9d03b10-399b-4f57-8790-a796d979ea3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:28.614435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1235/2457] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:55:28 2022] DELETE /identity/v3/projects/ebd6083e6f18454781ae160d105c568e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:28.620661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd3756-6307-4814-876d-aad175afd3d7 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:28.620955 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd3756-6307-4814-876d-aad175afd3d7 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:28.621236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd3756-6307-4814-876d-aad175afd3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:28.641413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-95fd3756-6307-4814-876d-aad175afd3d7 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:28.642643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-95fd3756-6307-4814-876d-aad175afd3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:28.708161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1223/2458] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:28.716855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c6ddc-8fd8-4896-8c1a-a736b7374e4f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:28.717174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c6ddc-8fd8-4896-8c1a-a736b7374e4f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:28.717494 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c6ddc-8fd8-4896-8c1a-a736b7374e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:28.738281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-ec9c6ddc-8fd8-4896-8c1a-a736b7374e4f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:28.739510 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ec9c6ddc-8fd8-4896-8c1a-a736b7374e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:28.748115 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1236/2459] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:28.753175 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd3756-6307-4814-876d-aad175afd3d7 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:28.769116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd3756-6307-4814-876d-aad175afd3d7 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:28.776152 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce231f-8941-4712-9dc8-a3d284049f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:28.777222 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce231f-8941-4712-9dc8-a3d284049f59 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:28.777699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce231f-8941-4712-9dc8-a3d284049f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:28.778161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce231f-8941-4712-9dc8-a3d284049f59 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:28.779405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce231f-8941-4712-9dc8-a3d284049f59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:55:28.779890 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce231f-8941-4712-9dc8-a3d284049f59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:28.783472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce231f-8941-4712-9dc8-a3d284049f59 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:28.803217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1224/2460] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:55:28.809795 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c6ddc-8fd8-4896-8c1a-a736b7374e4f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:28.821241 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c6ddc-8fd8-4896-8c1a-a736b7374e4f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:28.826148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ab39c-7b10-41d7-9d10-350a754cfc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:28.826898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ab39c-7b10-41d7-9d10-350a754cfc9f admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:28.827186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ab39c-7b10-41d7-9d10-350a754cfc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:28.827472 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ab39c-7b10-41d7-9d10-350a754cfc9f admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:28.828992 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ab39c-7b10-41d7-9d10-350a754cfc9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:28.832624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ab39c-7b10-41d7-9d10-350a754cfc9f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:28.874368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1237/2461] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:55:28 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:55:28.880845 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce231f-8941-4712-9dc8-a3d284049f59 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:28.899841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce231f-8941-4712-9dc8-a3d284049f59 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:28.962141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e8522-bb7f-4d63-a753-72cb5d0dea65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:28.962826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e8522-bb7f-4d63-a753-72cb5d0dea65 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:28.963110 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e8522-bb7f-4d63-a753-72cb5d0dea65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:28.963388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e8522-bb7f-4d63-a753-72cb5d0dea65 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:28.964103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e8522-bb7f-4d63-a753-72cb5d0dea65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:28.966047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e8522-bb7f-4d63-a753-72cb5d0dea65 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:28.976828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:28.992122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1225/2462] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:55:28 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:55:28.996529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ab39c-7b10-41d7-9d10-350a754cfc9f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:29.007460 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ab39c-7b10-41d7-9d10-350a754cfc9f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:29.012653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823aade-46da-4cb7-850a-723471269182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:29.013355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823aade-46da-4cb7-850a-723471269182 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:29.013643 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823aade-46da-4cb7-850a-723471269182 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:29.013922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823aade-46da-4cb7-850a-723471269182 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:29.014664 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823aade-46da-4cb7-850a-723471269182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:29.017142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823aade-46da-4cb7-850a-723471269182 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:29.027674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1238/2463] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:55:29.031913 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e8522-bb7f-4d63-a753-72cb5d0dea65 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:29.042533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e8522-bb7f-4d63-a753-72cb5d0dea65 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:29.047234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9eb511-5cbf-4e43-960d-2fbe41f38049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:29.047909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9eb511-5cbf-4e43-960d-2fbe41f38049 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:29.048345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9eb511-5cbf-4e43-960d-2fbe41f38049 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:29.048477 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9eb511-5cbf-4e43-960d-2fbe41f38049 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:29.049286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9eb511-5cbf-4e43-960d-2fbe41f38049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:29.050892 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9eb511-5cbf-4e43-960d-2fbe41f38049 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:29.061827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1226/2464] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:55:29.066723 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823aade-46da-4cb7-850a-723471269182 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:29.077323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823aade-46da-4cb7-850a-723471269182 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:29.082923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569de415-8f13-4984-ad4d-ec1825189ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:29.083830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569de415-8f13-4984-ad4d-ec1825189ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:29.084117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569de415-8f13-4984-ad4d-ec1825189ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:29.084398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569de415-8f13-4984-ad4d-ec1825189ff4 admin admin] PATH_INFO: `/v3/projects/6049d2c68f6241cfa404fabfe89c643d/users/7b9fc948fda44d14b61cf81ddfd657c2/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:29.086903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:29.102071 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569de415-8f13-4984-ad4d-ec1825189ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6049d2c68f6241cfa404fabfe89c643d, user_id=7b9fc948fda44d14b61cf81ddfd657c2, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:29.105442 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569de415-8f13-4984-ad4d-ec1825189ff4 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:29.120146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1239/2465] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:55:29 2022] PUT /identity/v3/projects/6049d2c68f6241cfa404fabfe89c643d/users/7b9fc948fda44d14b61cf81ddfd657c2/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:29.133243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160cfc8-afb1-44ab-8d6e-c02eff790078 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:29.133542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160cfc8-afb1-44ab-8d6e-c02eff790078 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:29.133855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160cfc8-afb1-44ab-8d6e-c02eff790078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:29.163209 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-2160cfc8-afb1-44ab-8d6e-c02eff790078 None None] MFA Rules not processed for user `7b9fc948fda44d14b61cf81ddfd657c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:29.164430 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2160cfc8-afb1-44ab-8d6e-c02eff790078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:29.235428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1227/2466] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:29.247097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569de415-8f13-4984-ad4d-ec1825189ff4 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:29.249054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-569de415-8f13-4984-ad4d-ec1825189ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:29.337531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569de415-8f13-4984-ad4d-ec1825189ff4 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:29.341403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2e4e4-348e-4271-86de-658bbcb75819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:29.342062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2e4e4-348e-4271-86de-658bbcb75819 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:29.342352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2e4e4-348e-4271-86de-658bbcb75819 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:29.342626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2e4e4-348e-4271-86de-658bbcb75819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:29.352399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2e4e4-348e-4271-86de-658bbcb75819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:29.353965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2e4e4-348e-4271-86de-658bbcb75819 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:29.364857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1240/2467] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:55:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:55:29.386705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160cfc8-afb1-44ab-8d6e-c02eff790078 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:29.453291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160cfc8-afb1-44ab-8d6e-c02eff790078 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:29.458031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:29.458922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:29.459202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:29.459470 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] PATH_INFO: `/v3/users/7b9fc948fda44d14b61cf81ddfd657c2` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:29.460720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b9fc948fda44d14b61cf81ddfd657c2)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:29.462631 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:29.512390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b9fc948fda44d14b61cf81ddfd657c2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:29.520972 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b9fc948fda44d14b61cf81ddfd657c2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:29.533674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b9fc948fda44d14b61cf81ddfd657c2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:29.535656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:29.545337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b9fc948fda44d14b61cf81ddfd657c2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:29.546151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b9fc948fda44d14b61cf81ddfd657c2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:29.553058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1228/2468] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:55:29 2022] DELETE /identity/v3/users/7b9fc948fda44d14b61cf81ddfd657c2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:29.557674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a0aaa-fa06-42ed-8620-0ae9bffc5077 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:29.558170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a0aaa-fa06-42ed-8620-0ae9bffc5077 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:29.558329 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a0aaa-fa06-42ed-8620-0ae9bffc5077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:29.578284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d06a0aaa-fa06-42ed-8620-0ae9bffc5077 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:29.579502 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d06a0aaa-fa06-42ed-8620-0ae9bffc5077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:29.638709 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1241/2469] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:29.648373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:29.649860 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:29.708789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6625c-c0b3-4a5f-96c9-e4554851f6c6 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:29.712513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef1a03-2f5c-48e8-b560-dd890336c4be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:29.713198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1a03-2f5c-48e8-b560-dd890336c4be service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:29.713483 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1a03-2f5c-48e8-b560-dd890336c4be service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:29.713757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1a03-2f5c-48e8-b560-dd890336c4be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:29.723581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef1a03-2f5c-48e8-b560-dd890336c4be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:29.725089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef1a03-2f5c-48e8-b560-dd890336c4be service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:29.735601 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1229/2470] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:55:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:55:30.023860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a0aaa-fa06-42ed-8620-0ae9bffc5077 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:30.025716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d06a0aaa-fa06-42ed-8620-0ae9bffc5077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:30.044809 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a0aaa-fa06-42ed-8620-0ae9bffc5077 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:30.051291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e17da-2665-4db4-a747-39e1c5abeb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:30.052710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e17da-2665-4db4-a747-39e1c5abeb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:30.052710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e17da-2665-4db4-a747-39e1c5abeb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:30.052933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e17da-2665-4db4-a747-39e1c5abeb3f admin admin] PATH_INFO: `/v3/projects/6049d2c68f6241cfa404fabfe89c643d` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:30.054193 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e17da-2665-4db4-a747-39e1c5abeb3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6049d2c68f6241cfa404fabfe89c643d)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:30.056137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e17da-2665-4db4-a747-39e1c5abeb3f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:30.106351 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-870e17da-2665-4db4-a747-39e1c5abeb3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6049d2c68f6241cfa404fabfe89c643d', 'request_id': 'req-870e17da-2665-4db4-a747-39e1c5abeb3f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:30.117598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-870e17da-2665-4db4-a747-39e1c5abeb3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6049d2c68f6241cfa404fabfe89c643d', 'request_id': 'req-870e17da-2665-4db4-a747-39e1c5abeb3f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:30.123423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:30.128461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-870e17da-2665-4db4-a747-39e1c5abeb3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6049d2c68f6241cfa404fabfe89c643d', 'request_id': 'req-870e17da-2665-4db4-a747-39e1c5abeb3f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:30.144736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-870e17da-2665-4db4-a747-39e1c5abeb3f admin admin] The token cache is being invalidate because project 6049d2c68f6241cfa404fabfe89c643d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:55:30.144965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-870e17da-2665-4db4-a747-39e1c5abeb3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:30.145634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-870e17da-2665-4db4-a747-39e1c5abeb3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:30.147246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1242/2471] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:55:30 2022] DELETE /identity/v3/projects/6049d2c68f6241cfa404fabfe89c643d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:30.153031 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15ac51-bbfa-4e3b-b107-3f4ddc2d4139 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:30.153855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15ac51-bbfa-4e3b-b107-3f4ddc2d4139 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:30.154258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15ac51-bbfa-4e3b-b107-3f4ddc2d4139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:30.186713 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ce15ac51-bbfa-4e3b-b107-3f4ddc2d4139 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:30.188855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ce15ac51-bbfa-4e3b-b107-3f4ddc2d4139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:30.276133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1230/2472] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:30.286062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15ee65-4969-4f23-ab43-2ae62a8592fe None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:30.286340 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15ee65-4969-4f23-ab43-2ae62a8592fe None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:30.286623 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15ee65-4969-4f23-ab43-2ae62a8592fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:30.308714 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-6e15ee65-4969-4f23-ab43-2ae62a8592fe None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:30.309960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6e15ee65-4969-4f23-ab43-2ae62a8592fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:30.318205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1243/2473] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:30.322704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15ac51-bbfa-4e3b-b107-3f4ddc2d4139 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:30.333220 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15ac51-bbfa-4e3b-b107-3f4ddc2d4139 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:30.338146 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ed3f2-c81a-454d-8cb3-596fe2b843ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:30.338832 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ed3f2-c81a-454d-8cb3-596fe2b843ac admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:30.339117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ed3f2-c81a-454d-8cb3-596fe2b843ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:30.339392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ed3f2-c81a-454d-8cb3-596fe2b843ac admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:30.340135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ed3f2-c81a-454d-8cb3-596fe2b843ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:55:30.340443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ed3f2-c81a-454d-8cb3-596fe2b843ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:30.342107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ed3f2-c81a-454d-8cb3-596fe2b843ac admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:30.355207 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1231/2474] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:55:30.360761 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15ee65-4969-4f23-ab43-2ae62a8592fe None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:30.371032 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15ee65-4969-4f23-ab43-2ae62a8592fe None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:30.376308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b2dc4-6918-497f-8523-a6998366ce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:30.377025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b2dc4-6918-497f-8523-a6998366ce7a admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:30.377308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b2dc4-6918-497f-8523-a6998366ce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:30.377582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b2dc4-6918-497f-8523-a6998366ce7a admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:30.378354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b2dc4-6918-497f-8523-a6998366ce7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:30.380275 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b2dc4-6918-497f-8523-a6998366ce7a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:30.413997 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1244/2475] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:55:30 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:55:30.415617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ed3f2-c81a-454d-8cb3-596fe2b843ac admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:30.426360 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ed3f2-c81a-454d-8cb3-596fe2b843ac admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:30.506260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c61f9-b6d3-4e4d-8d37-b951ec33b13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:30.507384 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c61f9-b6d3-4e4d-8d37-b951ec33b13d admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:30.507877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c61f9-b6d3-4e4d-8d37-b951ec33b13d admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:30.508351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c61f9-b6d3-4e4d-8d37-b951ec33b13d admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:30.509689 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c61f9-b6d3-4e4d-8d37-b951ec33b13d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:30.513308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c61f9-b6d3-4e4d-8d37-b951ec33b13d admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:30.543745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1232/2476] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:55:30 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:55:30.548192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b2dc4-6918-497f-8523-a6998366ce7a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:30.566087 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b2dc4-6918-497f-8523-a6998366ce7a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:30.574422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925dc71-6939-40a5-ae2e-29c0ecb18a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:30.575592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925dc71-6939-40a5-ae2e-29c0ecb18a3b admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:30.576095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925dc71-6939-40a5-ae2e-29c0ecb18a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:30.576554 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925dc71-6939-40a5-ae2e-29c0ecb18a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:30.578507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925dc71-6939-40a5-ae2e-29c0ecb18a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:30.581641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925dc71-6939-40a5-ae2e-29c0ecb18a3b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:30.594210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1245/2477] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:55:30.599149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c61f9-b6d3-4e4d-8d37-b951ec33b13d admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:30.613830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c61f9-b6d3-4e4d-8d37-b951ec33b13d admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:30.619433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fa4c9-ee0e-404a-8964-951ce7439de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:30.620386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fa4c9-ee0e-404a-8964-951ce7439de9 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:30.620702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fa4c9-ee0e-404a-8964-951ce7439de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:30.620986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fa4c9-ee0e-404a-8964-951ce7439de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:30.621737 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fa4c9-ee0e-404a-8964-951ce7439de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:30.623591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fa4c9-ee0e-404a-8964-951ce7439de9 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:30.630702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:30.637329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1233/2478] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:55:30.642432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925dc71-6939-40a5-ae2e-29c0ecb18a3b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:30.653237 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925dc71-6939-40a5-ae2e-29c0ecb18a3b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:30.659179 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4e3e8-732b-42b7-90bd-027e9f2c794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:30.660102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4e3e8-732b-42b7-90bd-027e9f2c794c admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:30.660382 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4e3e8-732b-42b7-90bd-027e9f2c794c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:30.660708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4e3e8-732b-42b7-90bd-027e9f2c794c admin admin] PATH_INFO: `/v3/projects/fedc6121a35641b4bada1727a446619b/users/9c23346aaf044c13830ab5c9ed51c241/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:30.677772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4e3e8-732b-42b7-90bd-027e9f2c794c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fedc6121a35641b4bada1727a446619b, user_id=9c23346aaf044c13830ab5c9ed51c241, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:30.680434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4e3e8-732b-42b7-90bd-027e9f2c794c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:30.700596 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1246/2479] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:55:30 2022] PUT /identity/v3/projects/fedc6121a35641b4bada1727a446619b/users/9c23346aaf044c13830ab5c9ed51c241/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:30.712387 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5388ca6-68a6-4b57-9cb7-392669b157cf None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:30.712718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5388ca6-68a6-4b57-9cb7-392669b157cf None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:30.712997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5388ca6-68a6-4b57-9cb7-392669b157cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:30.747735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-d5388ca6-68a6-4b57-9cb7-392669b157cf None None] MFA Rules not processed for user `9c23346aaf044c13830ab5c9ed51c241`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:30.749144 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d5388ca6-68a6-4b57-9cb7-392669b157cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:30.829581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1234/2480] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:30.842536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4e3e8-732b-42b7-90bd-027e9f2c794c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:30.844283 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a1a4e3e8-732b-42b7-90bd-027e9f2c794c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:30.910888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4e3e8-732b-42b7-90bd-027e9f2c794c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:30.914672 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf24100-4f02-4168-abe0-e2a704da8ae9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '925bf2da4df74e86aa84ec62a267d088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:30.915341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf24100-4f02-4168-abe0-e2a704da8ae9 service glance] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:30.915629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf24100-4f02-4168-abe0-e2a704da8ae9 service glance] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:30.915907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf24100-4f02-4168-abe0-e2a704da8ae9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:30.925884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf24100-4f02-4168-abe0-e2a704da8ae9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:30.927346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf24100-4f02-4168-abe0-e2a704da8ae9 service glance] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:30.938341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1247/2481] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:55:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:55:40.825406 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=32 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:40.932953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:DELETE=1,keystone:SELECT=57,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:46.942441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5388ca6-68a6-4b57-9cb7-392669b157cf None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:46.944114 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d5388ca6-68a6-4b57-9cb7-392669b157cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:47.016433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5388ca6-68a6-4b57-9cb7-392669b157cf None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:47.022270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba6657-70f2-4924-808e-34eff16381d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:47.023080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba6657-70f2-4924-808e-34eff16381d9 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:47.023428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba6657-70f2-4924-808e-34eff16381d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:47.023773 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba6657-70f2-4924-808e-34eff16381d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:47.025974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-53ba6657-70f2-4924-808e-34eff16381d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:47.103764 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba6657-70f2-4924-808e-34eff16381d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:47.105262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba6657-70f2-4924-808e-34eff16381d9 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:47.118182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1235/2482] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:55:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:55:48.306238 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf24100-4f02-4168-abe0-e2a704da8ae9 service glance] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:48.317357 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf24100-4f02-4168-abe0-e2a704da8ae9 service glance] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:48.321115 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767ef97-7bf7-487b-bca8-1b53d285c425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:48.321766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767ef97-7bf7-487b-bca8-1b53d285c425 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:48.322055 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767ef97-7bf7-487b-bca8-1b53d285c425 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:48.322500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767ef97-7bf7-487b-bca8-1b53d285c425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:48.324349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7767ef97-7bf7-487b-bca8-1b53d285c425 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:48.338906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767ef97-7bf7-487b-bca8-1b53d285c425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:48.340376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767ef97-7bf7-487b-bca8-1b53d285c425 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:48.352000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1248/2483] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:55:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:55:52.714047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd7fb7-1178-446c-9674-b1c2f67f3eeb None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:52.714620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd7fb7-1178-446c-9674-b1c2f67f3eeb None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:52.714620 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd7fb7-1178-446c-9674-b1c2f67f3eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:52.763009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-09cd7fb7-1178-446c-9674-b1c2f67f3eeb None None] MFA Rules not processed for user `8e682232b7374450a7265e092602a5a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:52.764255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-09cd7fb7-1178-446c-9674-b1c2f67f3eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:52.823104 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1236/2484] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:58.255743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767ef97-7bf7-487b-bca8-1b53d285c425 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:58.324434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767ef97-7bf7-487b-bca8-1b53d285c425 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:58.329166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:58.330052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:58.330337 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:58.330619 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] PATH_INFO: `/v3/users/9c23346aaf044c13830ab5c9ed51c241` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:58.331843 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c23346aaf044c13830ab5c9ed51c241)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:58.333895 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:58.385397 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c23346aaf044c13830ab5c9ed51c241'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:58.386102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c23346aaf044c13830ab5c9ed51c241'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:58.394174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c23346aaf044c13830ab5c9ed51c241'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:58.407651 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c23346aaf044c13830ab5c9ed51c241'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:58.414975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c23346aaf044c13830ab5c9ed51c241'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:58.421953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1249/2485] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:55:58 2022] DELETE /identity/v3/users/9c23346aaf044c13830ab5c9ed51c241 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:55:58.426693 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d382080-f307-4dea-92e4-9a7cc98b4c25 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:58.427191 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d382080-f307-4dea-92e4-9a7cc98b4c25 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:58.427312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d382080-f307-4dea-92e4-9a7cc98b4c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:58.446762 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-3d382080-f307-4dea-92e4-9a7cc98b4c25 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:58.447985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3d382080-f307-4dea-92e4-9a7cc98b4c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:58.471628 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:58.512313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1237/2486] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:58.522951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:58.524451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:58.590419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e27e0-920a-4d09-b2b7-d6827e3b6c79 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:58.595012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131de283-7979-4dd9-913f-1eee312d9b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:58.596482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131de283-7979-4dd9-913f-1eee312d9b7d service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:58.596482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131de283-7979-4dd9-913f-1eee312d9b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:58.596482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131de283-7979-4dd9-913f-1eee312d9b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:58.607105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131de283-7979-4dd9-913f-1eee312d9b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:58.609157 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131de283-7979-4dd9-913f-1eee312d9b7d service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:58.620924 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1250/2487] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:55:58.937883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d382080-f307-4dea-92e4-9a7cc98b4c25 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:58.939341 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3d382080-f307-4dea-92e4-9a7cc98b4c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:58.954575 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d382080-f307-4dea-92e4-9a7cc98b4c25 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:58.960643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:58.961679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:58.961970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:58.962250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f admin admin] PATH_INFO: `/v3/projects/fedc6121a35641b4bada1727a446619b` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:58.963671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fedc6121a35641b4bada1727a446619b)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:58.966511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:59.022968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fedc6121a35641b4bada1727a446619b', 'request_id': 'req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:59.032058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fedc6121a35641b4bada1727a446619b', 'request_id': 'req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:59.038237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fedc6121a35641b4bada1727a446619b', 'request_id': 'req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:59.050377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f admin admin] The token cache is being invalidate because project fedc6121a35641b4bada1727a446619b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:55:59.050670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:59.051348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-eb2dc2ca-04a4-44f0-9eb4-d7ba0f7cae3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:55:59.053116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1238/2488] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:55:58 2022] DELETE /identity/v3/projects/fedc6121a35641b4bada1727a446619b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:55:59.058819 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad243b-ebd8-4817-80ad-7de6d1a3d676 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:59.059107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad243b-ebd8-4817-80ad-7de6d1a3d676 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:59.059399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad243b-ebd8-4817-80ad-7de6d1a3d676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:59.087095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-6bad243b-ebd8-4817-80ad-7de6d1a3d676 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:59.088343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-6bad243b-ebd8-4817-80ad-7de6d1a3d676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:59.093103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:59.158043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1251/2489] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:59.166228 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4aaf4-6534-4b3b-87b5-2fde1888f5bf None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:59.166522 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4aaf4-6534-4b3b-87b5-2fde1888f5bf None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:59.166810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4aaf4-6534-4b3b-87b5-2fde1888f5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:59.188048 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-25d4aaf4-6534-4b3b-87b5-2fde1888f5bf None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:59.189220 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-25d4aaf4-6534-4b3b-87b5-2fde1888f5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:59.197263 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1239/2490] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:55:59.202370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad243b-ebd8-4817-80ad-7de6d1a3d676 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:59.212686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad243b-ebd8-4817-80ad-7de6d1a3d676 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:59.217423 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdf269-9e2f-4e00-b40c-945d365b74af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:59.218100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdf269-9e2f-4e00-b40c-945d365b74af admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:59.218383 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdf269-9e2f-4e00-b40c-945d365b74af admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:59.218661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdf269-9e2f-4e00-b40c-945d365b74af admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:59.219398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdf269-9e2f-4e00-b40c-945d365b74af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:55:59.219706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdf269-9e2f-4e00-b40c-945d365b74af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:59.221355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdf269-9e2f-4e00-b40c-945d365b74af admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:59.233670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1252/2491] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:55:59.238136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4aaf4-6534-4b3b-87b5-2fde1888f5bf None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:59.248126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4aaf4-6534-4b3b-87b5-2fde1888f5bf None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:59.253441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e3f2f-3509-4f7e-8e09-0af9046ff4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:59.254121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e3f2f-3509-4f7e-8e09-0af9046ff4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:59.254409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e3f2f-3509-4f7e-8e09-0af9046ff4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:59.254721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e3f2f-3509-4f7e-8e09-0af9046ff4f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:59.255580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e3f2f-3509-4f7e-8e09-0af9046ff4f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:59.257656 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e3f2f-3509-4f7e-8e09-0af9046ff4f4 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:59.302320 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1240/2492] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:55:59 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:55:59.308518 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdf269-9e2f-4e00-b40c-945d365b74af admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:59.321544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdf269-9e2f-4e00-b40c-945d365b74af admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:59.394860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d81b9-9e18-4f99-bda7-d30c33da1cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:59.395559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d81b9-9e18-4f99-bda7-d30c33da1cec admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:59.396032 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d81b9-9e18-4f99-bda7-d30c33da1cec admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:59.396311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d81b9-9e18-4f99-bda7-d30c33da1cec admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:59.397100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d81b9-9e18-4f99-bda7-d30c33da1cec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:59.399594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d81b9-9e18-4f99-bda7-d30c33da1cec admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:59.426956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1253/2493] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:55:59 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:55:59.434170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e3f2f-3509-4f7e-8e09-0af9046ff4f4 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:59.453732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e3f2f-3509-4f7e-8e09-0af9046ff4f4 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:59.459706 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc73fa4-1aba-412d-bc1a-89161dd8377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:59.460497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc73fa4-1aba-412d-bc1a-89161dd8377e admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:59.460824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc73fa4-1aba-412d-bc1a-89161dd8377e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:59.461112 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc73fa4-1aba-412d-bc1a-89161dd8377e admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:59.461903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc73fa4-1aba-412d-bc1a-89161dd8377e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:59.463600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc73fa4-1aba-412d-bc1a-89161dd8377e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:59.477158 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1241/2494] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:55:59.481734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d81b9-9e18-4f99-bda7-d30c33da1cec admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:59.497851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d81b9-9e18-4f99-bda7-d30c33da1cec admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:59.505567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748b56-6216-4e35-9517-3872a864b144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:59.506741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748b56-6216-4e35-9517-3872a864b144 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:59.507309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748b56-6216-4e35-9517-3872a864b144 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:59.507850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748b56-6216-4e35-9517-3872a864b144 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:59.509106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748b56-6216-4e35-9517-3872a864b144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:59.511087 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748b56-6216-4e35-9517-3872a864b144 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:59.523593 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1254/2495] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:55:59.527959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc73fa4-1aba-412d-bc1a-89161dd8377e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:59.539562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc73fa4-1aba-412d-bc1a-89161dd8377e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:59.544738 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbd24d-4dd0-4005-b6c0-53310d523eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:59.546230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbd24d-4dd0-4005-b6c0-53310d523eab admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:59.546518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbd24d-4dd0-4005-b6c0-53310d523eab admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:59.546797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbd24d-4dd0-4005-b6c0-53310d523eab admin admin] PATH_INFO: `/v3/projects/ae363f53121a49908aa9dde843932ecf/users/1a57c27568474dbe97b938e8e827960f/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:59.565899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbd24d-4dd0-4005-b6c0-53310d523eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae363f53121a49908aa9dde843932ecf, user_id=1a57c27568474dbe97b938e8e827960f, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:59.568685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbd24d-4dd0-4005-b6c0-53310d523eab admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:59.584753 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1242/2496] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:55:59 2022] PUT /identity/v3/projects/ae363f53121a49908aa9dde843932ecf/users/1a57c27568474dbe97b938e8e827960f/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:55:59.595956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c39a0b-c396-4add-ba6b-e89d2c3da570 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:59.596409 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c39a0b-c396-4add-ba6b-e89d2c3da570 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:59.596786 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c39a0b-c396-4add-ba6b-e89d2c3da570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:59.630851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c8c39a0b-c396-4add-ba6b-e89d2c3da570 None None] MFA Rules not processed for user `1a57c27568474dbe97b938e8e827960f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:55:59.632162 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c8c39a0b-c396-4add-ba6b-e89d2c3da570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:55:59.717302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1255/2497] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:55:59.721873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbd24d-4dd0-4005-b6c0-53310d523eab admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:59.772279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:59.799338 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbd24d-4dd0-4005-b6c0-53310d523eab admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:59.808403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e85b4-645b-4337-a893-ddc80807386b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:59.809146 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e85b4-645b-4337-a893-ddc80807386b admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:59.809447 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e85b4-645b-4337-a893-ddc80807386b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:59.809728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e85b4-645b-4337-a893-ddc80807386b admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:59.810490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e85b4-645b-4337-a893-ddc80807386b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:59.812512 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e85b4-645b-4337-a893-ddc80807386b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:59.851962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1243/2498] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:55:59 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:55:59.854194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c39a0b-c396-4add-ba6b-e89d2c3da570 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:55:59.863097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:55:59.868126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c39a0b-c396-4add-ba6b-e89d2c3da570 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:55:59.944388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9836ebb-3959-46d1-a4b3-4d954038dd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:55:59.945320 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836ebb-3959-46d1-a4b3-4d954038dd83 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:55:59.945814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836ebb-3959-46d1-a4b3-4d954038dd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:55:59.946264 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836ebb-3959-46d1-a4b3-4d954038dd83 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:55:59.947566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9836ebb-3959-46d1-a4b3-4d954038dd83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:55:59.950868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9836ebb-3959-46d1-a4b3-4d954038dd83 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:55:59.979324 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1256/2499] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:55:59 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:55:59.983998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e85b4-645b-4337-a893-ddc80807386b admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:00.001611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e85b4-645b-4337-a893-ddc80807386b admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:00.010367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f567547-528f-40d4-ba9c-8e2faffd2926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:00.011420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f567547-528f-40d4-ba9c-8e2faffd2926 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:00.011707 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f567547-528f-40d4-ba9c-8e2faffd2926 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:00.012057 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f567547-528f-40d4-ba9c-8e2faffd2926 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:00.012848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f567547-528f-40d4-ba9c-8e2faffd2926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:00.015068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f567547-528f-40d4-ba9c-8e2faffd2926 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:00.033775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1244/2500] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:56:00.039195 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9836ebb-3959-46d1-a4b3-4d954038dd83 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:00.055370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9836ebb-3959-46d1-a4b3-4d954038dd83 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:00.060505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd79e3-965a-44c9-bf6a-2fb4202fe79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:00.061256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd79e3-965a-44c9-bf6a-2fb4202fe79f admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:00.061546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd79e3-965a-44c9-bf6a-2fb4202fe79f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:00.061831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd79e3-965a-44c9-bf6a-2fb4202fe79f admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:00.062664 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd79e3-965a-44c9-bf6a-2fb4202fe79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:00.064472 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd79e3-965a-44c9-bf6a-2fb4202fe79f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:00.076080 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1257/2501] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:56:00.080621 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f567547-528f-40d4-ba9c-8e2faffd2926 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:00.091198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f567547-528f-40d4-ba9c-8e2faffd2926 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:00.096550 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738598a-7f38-47c6-868a-42b704c19008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:00.097478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738598a-7f38-47c6-868a-42b704c19008 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:00.097758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738598a-7f38-47c6-868a-42b704c19008 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:00.098036 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738598a-7f38-47c6-868a-42b704c19008 admin admin] PATH_INFO: `/v3/projects/a856c08bea534c26ac46e5e878ba10dc/users/f40f791e7a23428fa0200001bc6369d8/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:00.114880 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738598a-7f38-47c6-868a-42b704c19008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a856c08bea534c26ac46e5e878ba10dc, user_id=f40f791e7a23428fa0200001bc6369d8, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:00.117611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738598a-7f38-47c6-868a-42b704c19008 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:00.131842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1245/2502] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:56:00 2022] PUT /identity/v3/projects/a856c08bea534c26ac46e5e878ba10dc/users/f40f791e7a23428fa0200001bc6369d8/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:56:00.143732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7c925-83d7-4270-a79b-297147537824 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:00.144019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7c925-83d7-4270-a79b-297147537824 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:00.144288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7c925-83d7-4270-a79b-297147537824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:00.175615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-0fd7c925-83d7-4270-a79b-297147537824 None None] MFA Rules not processed for user `f40f791e7a23428fa0200001bc6369d8`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:56:00.176897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0fd7c925-83d7-4270-a79b-297147537824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:00.250274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1258/2503] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:56:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:56:00.255090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738598a-7f38-47c6-868a-42b704c19008 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:00.328304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738598a-7f38-47c6-868a-42b704c19008 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:00.333635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c58f5-2275-4880-aa9b-10e683feab57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:00.334336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c58f5-2275-4880-aa9b-10e683feab57 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:00.334622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c58f5-2275-4880-aa9b-10e683feab57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:00.334903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c58f5-2275-4880-aa9b-10e683feab57 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:00.335627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c58f5-2275-4880-aa9b-10e683feab57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:00.337634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c58f5-2275-4880-aa9b-10e683feab57 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:00.367949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1246/2504] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:56:00 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:56:00.372630 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7c925-83d7-4270-a79b-297147537824 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:00.384456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7c925-83d7-4270-a79b-297147537824 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:00.455544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a44b0-f296-400f-9803-3e7d5012d326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:00.456255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a44b0-f296-400f-9803-3e7d5012d326 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:00.456544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a44b0-f296-400f-9803-3e7d5012d326 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:00.456868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a44b0-f296-400f-9803-3e7d5012d326 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:00.457601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a44b0-f296-400f-9803-3e7d5012d326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:00.459589 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a44b0-f296-400f-9803-3e7d5012d326 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:00.486052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1259/2505] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:56:00 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:56:00.490212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c58f5-2275-4880-aa9b-10e683feab57 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:00.502221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c58f5-2275-4880-aa9b-10e683feab57 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:00.507135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c51964-a76e-4522-9a1c-8aaf24bab6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:00.507828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c51964-a76e-4522-9a1c-8aaf24bab6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:00.508133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c51964-a76e-4522-9a1c-8aaf24bab6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:00.508411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c51964-a76e-4522-9a1c-8aaf24bab6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:00.509188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c51964-a76e-4522-9a1c-8aaf24bab6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:00.510956 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c51964-a76e-4522-9a1c-8aaf24bab6d7 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:00.524715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1247/2506] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:56:00.528901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a44b0-f296-400f-9803-3e7d5012d326 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:00.539521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:56:00.542789 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a44b0-f296-400f-9803-3e7d5012d326 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:00.549070 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657637a-a355-4cd0-9038-c4da6312c1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:00.550461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657637a-a355-4cd0-9038-c4da6312c1e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:00.550784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657637a-a355-4cd0-9038-c4da6312c1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:00.551742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657637a-a355-4cd0-9038-c4da6312c1e2 admin admin] PATH_INFO: `/v3/projects/bd58e8edc7f54306a4e1bc333844e5bd/users/1468acd925ca40acbda442c1a6615735/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:00.569598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657637a-a355-4cd0-9038-c4da6312c1e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd58e8edc7f54306a4e1bc333844e5bd, user_id=1468acd925ca40acbda442c1a6615735, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:00.572483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657637a-a355-4cd0-9038-c4da6312c1e2 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:00.594405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1260/2507] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:56:00 2022] PUT /identity/v3/projects/bd58e8edc7f54306a4e1bc333844e5bd/users/1468acd925ca40acbda442c1a6615735/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:56:00.598619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c51964-a76e-4522-9a1c-8aaf24bab6d7 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:00.609679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c51964-a76e-4522-9a1c-8aaf24bab6d7 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:00.697641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ea3bc-2178-4e0c-94d9-d50ed00f56df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:00.698359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ea3bc-2178-4e0c-94d9-d50ed00f56df admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:00.698645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ea3bc-2178-4e0c-94d9-d50ed00f56df admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:00.699103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ea3bc-2178-4e0c-94d9-d50ed00f56df admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:00.699702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ea3bc-2178-4e0c-94d9-d50ed00f56df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:00.701465 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ea3bc-2178-4e0c-94d9-d50ed00f56df admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:00.709021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:56:00.715006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1248/2508] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:56:00.724820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657637a-a355-4cd0-9038-c4da6312c1e2 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:00.734727 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657637a-a355-4cd0-9038-c4da6312c1e2 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:00.741076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aacbce-5f80-40a2-9194-0c2c5711c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:00.741999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aacbce-5f80-40a2-9194-0c2c5711c23f admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:00.742441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aacbce-5f80-40a2-9194-0c2c5711c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:00.742730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aacbce-5f80-40a2-9194-0c2c5711c23f admin admin] PATH_INFO: `/v3/projects/bd58e8edc7f54306a4e1bc333844e5bd/users/1468acd925ca40acbda442c1a6615735/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:00.745267 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aacbce-5f80-40a2-9194-0c2c5711c23f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd58e8edc7f54306a4e1bc333844e5bd, user_id=1468acd925ca40acbda442c1a6615735, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:00.748051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aacbce-5f80-40a2-9194-0c2c5711c23f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:00.761942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1261/2509] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:56:00 2022] PUT /identity/v3/projects/bd58e8edc7f54306a4e1bc333844e5bd/users/1468acd925ca40acbda442c1a6615735/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:56:00.835396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda954-4a6b-4572-a88c-2c2b4c67e1b5 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:00.835702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda954-4a6b-4572-a88c-2c2b4c67e1b5 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:00.836070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda954-4a6b-4572-a88c-2c2b4c67e1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:00.877779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-16cda954-4a6b-4572-a88c-2c2b4c67e1b5 None None] MFA Rules not processed for user `1468acd925ca40acbda442c1a6615735`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:56:00.879107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-16cda954-4a6b-4572-a88c-2c2b4c67e1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:00.994454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1249/2510] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:56:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:56:01.005078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aacbce-5f80-40a2-9194-0c2c5711c23f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:01.007565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-59aacbce-5f80-40a2-9194-0c2c5711c23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:01.081722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aacbce-5f80-40a2-9194-0c2c5711c23f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:01.085365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108d23b-4798-4ac6-bf5c-0266493c8319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:01.086027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108d23b-4798-4ac6-bf5c-0266493c8319 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:01.086313 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108d23b-4798-4ac6-bf5c-0266493c8319 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:01.086587 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108d23b-4798-4ac6-bf5c-0266493c8319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:01.149098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108d23b-4798-4ac6-bf5c-0266493c8319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:01.150400 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108d23b-4798-4ac6-bf5c-0266493c8319 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:01.161009 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1262/2511] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:56:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:56:02.736968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cda954-4a6b-4572-a88c-2c2b4c67e1b5 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:02.742174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-16cda954-4a6b-4572-a88c-2c2b4c67e1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:02.756760 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cda954-4a6b-4572-a88c-2c2b4c67e1b5 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:02.763085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a196271-7193-41dc-8c72-dba36c9b6ba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:02.764229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a196271-7193-41dc-8c72-dba36c9b6ba3 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:02.764791 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a196271-7193-41dc-8c72-dba36c9b6ba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:02.765304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a196271-7193-41dc-8c72-dba36c9b6ba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:02.844532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a196271-7193-41dc-8c72-dba36c9b6ba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:02.845967 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a196271-7193-41dc-8c72-dba36c9b6ba3 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:02.856144 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1250/2512] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:56:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:56:03.503779 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108d23b-4798-4ac6-bf5c-0266493c8319 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:03.512860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108d23b-4798-4ac6-bf5c-0266493c8319 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:03.517182 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d09181-f18f-476e-88f2-4bdbccb10cc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:03.517939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d09181-f18f-476e-88f2-4bdbccb10cc3 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:03.518269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d09181-f18f-476e-88f2-4bdbccb10cc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:03.519161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d09181-f18f-476e-88f2-4bdbccb10cc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:03.528838 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d09181-f18f-476e-88f2-4bdbccb10cc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:03.530880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d09181-f18f-476e-88f2-4bdbccb10cc3 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:03.543422 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1263/2513] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:56:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:56:07.497563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a196271-7193-41dc-8c72-dba36c9b6ba3 service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:07.573400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a196271-7193-41dc-8c72-dba36c9b6ba3 service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:07.579029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:07.579935 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:07.580220 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:07.580497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] PATH_INFO: `/v3/users/1a57c27568474dbe97b938e8e827960f` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:07.581796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a57c27568474dbe97b938e8e827960f)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:07.584010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:07.620390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:56:07.649662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a57c27568474dbe97b938e8e827960f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:07.660583 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a57c27568474dbe97b938e8e827960f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:07.674479 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a57c27568474dbe97b938e8e827960f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:07.681555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a57c27568474dbe97b938e8e827960f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:07.682313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a57c27568474dbe97b938e8e827960f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:07.689140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1251/2514] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:56:07 2022] DELETE /identity/v3/users/1a57c27568474dbe97b938e8e827960f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:56:07.693811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c2862-8742-4384-b295-b79d9db6da9f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:07.694407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c2862-8742-4384-b295-b79d9db6da9f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:07.694407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c2862-8742-4384-b295-b79d9db6da9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:07.714337 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4f8c2862-8742-4384-b295-b79d9db6da9f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:56:07.715578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4f8c2862-8742-4384-b295-b79d9db6da9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:07.785030 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1264/2515] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:56:07.791728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:07.793310 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:07.851619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e933ea-1061-427e-ae74-29a77a4327da admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:07.855422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5929216-2e29-4911-9bcb-bd75bbd6a588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:07.856079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5929216-2e29-4911-9bcb-bd75bbd6a588 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:07.856360 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5929216-2e29-4911-9bcb-bd75bbd6a588 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:07.856670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5929216-2e29-4911-9bcb-bd75bbd6a588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:07.867149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5929216-2e29-4911-9bcb-bd75bbd6a588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:07.868884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5929216-2e29-4911-9bcb-bd75bbd6a588 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:07.879826 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1252/2516] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:56:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:56:08.174248 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c2862-8742-4384-b295-b79d9db6da9f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:08.175864 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4f8c2862-8742-4384-b295-b79d9db6da9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:08.188801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c2862-8742-4384-b295-b79d9db6da9f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:08.193629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:08.194533 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:08.195190 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:08.195190 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] PATH_INFO: `/v3/projects/ae363f53121a49908aa9dde843932ecf` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:08.197182 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae363f53121a49908aa9dde843932ecf)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:08.198314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:08.200253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:56:08.261260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae363f53121a49908aa9dde843932ecf', 'request_id': 'req-7fd46c90-119e-4c30-ab53-ba004fb6a045'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.271217 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae363f53121a49908aa9dde843932ecf', 'request_id': 'req-7fd46c90-119e-4c30-ab53-ba004fb6a045'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.278202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae363f53121a49908aa9dde843932ecf', 'request_id': 'req-7fd46c90-119e-4c30-ab53-ba004fb6a045'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.292705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] The token cache is being invalidate because project ae363f53121a49908aa9dde843932ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:56:08.293019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.293667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.295513 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1265/2517] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:56:08 2022] DELETE /identity/v3/projects/ae363f53121a49908aa9dde843932ecf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:56:08.299869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5929216-2e29-4911-9bcb-bd75bbd6a588 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:08.309939 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5929216-2e29-4911-9bcb-bd75bbd6a588 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:08.367145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:08.368022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:08.368302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:08.368848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] PATH_INFO: `/v3/users/f40f791e7a23428fa0200001bc6369d8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:08.370060 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f40f791e7a23428fa0200001bc6369d8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:08.371984 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:08.418725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:56:08.420412 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f40f791e7a23428fa0200001bc6369d8'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.428277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f40f791e7a23428fa0200001bc6369d8'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.441997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f40f791e7a23428fa0200001bc6369d8'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.449002 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f40f791e7a23428fa0200001bc6369d8'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.449720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f40f791e7a23428fa0200001bc6369d8'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.456561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1253/2518] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:56:08 2022] DELETE /identity/v3/users/f40f791e7a23428fa0200001bc6369d8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:56:08.741274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:08.743319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:08.760095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd46c90-119e-4c30-ab53-ba004fb6a045 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:08.835780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:08.836628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:08.836912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:08.837185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] PATH_INFO: `/v3/projects/a856c08bea534c26ac46e5e878ba10dc` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:08.838417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a856c08bea534c26ac46e5e878ba10dc)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:08.840345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:08.889000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a856c08bea534c26ac46e5e878ba10dc', 'request_id': 'req-d18fafb9-f434-4e3a-a587-0129ee62a494'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.897556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a856c08bea534c26ac46e5e878ba10dc', 'request_id': 'req-d18fafb9-f434-4e3a-a587-0129ee62a494'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.903349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a856c08bea534c26ac46e5e878ba10dc', 'request_id': 'req-d18fafb9-f434-4e3a-a587-0129ee62a494'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.915105 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] The token cache is being invalidate because project a856c08bea534c26ac46e5e878ba10dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:56:08.915403 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.916073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:08.917780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1266/2519] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:56:08 2022] DELETE /identity/v3/projects/a856c08bea534c26ac46e5e878ba10dc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:56:08.921895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:08.923346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:08.936461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdaf32-adb5-4e4c-9c88-6fbff4b0132e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:09.003079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3d202-5cbd-4fbf-80ef-ce0e36c78639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:09.003964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3d202-5cbd-4fbf-80ef-ce0e36c78639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:09.004251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3d202-5cbd-4fbf-80ef-ce0e36c78639 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:09.004531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3d202-5cbd-4fbf-80ef-ce0e36c78639 admin admin] PATH_INFO: `/v3/users/1468acd925ca40acbda442c1a6615735` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:09.005784 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3d202-5cbd-4fbf-80ef-ce0e36c78639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1468acd925ca40acbda442c1a6615735)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:09.007736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3d202-5cbd-4fbf-80ef-ce0e36c78639 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:09.059092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9eb3d202-5cbd-4fbf-80ef-ce0e36c78639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1468acd925ca40acbda442c1a6615735'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:09.072844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9eb3d202-5cbd-4fbf-80ef-ce0e36c78639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1468acd925ca40acbda442c1a6615735'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:09.086647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9eb3d202-5cbd-4fbf-80ef-ce0e36c78639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1468acd925ca40acbda442c1a6615735'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:09.094006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9eb3d202-5cbd-4fbf-80ef-ce0e36c78639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1468acd925ca40acbda442c1a6615735'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:09.094759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9eb3d202-5cbd-4fbf-80ef-ce0e36c78639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1468acd925ca40acbda442c1a6615735'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:09.101390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1254/2520] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:56:08 2022] DELETE /identity/v3/users/1468acd925ca40acbda442c1a6615735 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:56:09.390168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:09.391714 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:09.404931 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fafb9-f434-4e3a-a587-0129ee62a494 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:09.425283 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:56:09.469804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1ecf2-dbfa-440f-a0f8-bb359be65413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:09.470690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1ecf2-dbfa-440f-a0f8-bb359be65413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:09.470975 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1ecf2-dbfa-440f-a0f8-bb359be65413 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:09.471253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1ecf2-dbfa-440f-a0f8-bb359be65413 admin admin] PATH_INFO: `/v3/projects/bd58e8edc7f54306a4e1bc333844e5bd` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:09.472484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1ecf2-dbfa-440f-a0f8-bb359be65413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd58e8edc7f54306a4e1bc333844e5bd)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:09.474785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1ecf2-dbfa-440f-a0f8-bb359be65413 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:09.522614 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-97a1ecf2-dbfa-440f-a0f8-bb359be65413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd58e8edc7f54306a4e1bc333844e5bd', 'request_id': 'req-97a1ecf2-dbfa-440f-a0f8-bb359be65413'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:09.531090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-97a1ecf2-dbfa-440f-a0f8-bb359be65413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd58e8edc7f54306a4e1bc333844e5bd', 'request_id': 'req-97a1ecf2-dbfa-440f-a0f8-bb359be65413'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:09.537631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-97a1ecf2-dbfa-440f-a0f8-bb359be65413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd58e8edc7f54306a4e1bc333844e5bd', 'request_id': 'req-97a1ecf2-dbfa-440f-a0f8-bb359be65413'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:09.549822 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-97a1ecf2-dbfa-440f-a0f8-bb359be65413 admin admin] The token cache is being invalidate because project bd58e8edc7f54306a4e1bc333844e5bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:56:09.550111 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-97a1ecf2-dbfa-440f-a0f8-bb359be65413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:09.550778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-97a1ecf2-dbfa-440f-a0f8-bb359be65413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:09.552348 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1267/2521] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:56:09 2022] DELETE /identity/v3/projects/bd58e8edc7f54306a4e1bc333844e5bd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:56:09.557301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b4e5c-e91c-463f-9156-1e899e79d110 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:09.557596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b4e5c-e91c-463f-9156-1e899e79d110 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:09.557877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b4e5c-e91c-463f-9156-1e899e79d110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:09.579181 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ff2b4e5c-e91c-463f-9156-1e899e79d110 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:56:09.580401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ff2b4e5c-e91c-463f-9156-1e899e79d110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:09.622863 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:56:09.645443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1255/2522] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:56:09.653317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee31548-e866-4607-97c2-184fb754edf4 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:09.653610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee31548-e866-4607-97c2-184fb754edf4 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:09.653894 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee31548-e866-4607-97c2-184fb754edf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:09.676242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-0ee31548-e866-4607-97c2-184fb754edf4 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:56:09.677688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0ee31548-e866-4607-97c2-184fb754edf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:09.685892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1268/2523] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:56:09.690210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b4e5c-e91c-463f-9156-1e899e79d110 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:09.700134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b4e5c-e91c-463f-9156-1e899e79d110 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:09.704878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd543a2a-de07-493b-980b-d2c9e405e598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:09.705525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd543a2a-de07-493b-980b-d2c9e405e598 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:09.705808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd543a2a-de07-493b-980b-d2c9e405e598 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:09.706081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd543a2a-de07-493b-980b-d2c9e405e598 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:09.706848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd543a2a-de07-493b-980b-d2c9e405e598 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:56:09.707154 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd543a2a-de07-493b-980b-d2c9e405e598 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:09.708782 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd543a2a-de07-493b-980b-d2c9e405e598 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:09.721986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1256/2524] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:56:09.728334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee31548-e866-4607-97c2-184fb754edf4 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:09.740047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee31548-e866-4607-97c2-184fb754edf4 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:09.745181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe6e1e-4f25-4f28-bcbc-2a49276eac44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:09.745849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe6e1e-4f25-4f28-bcbc-2a49276eac44 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:09.746127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe6e1e-4f25-4f28-bcbc-2a49276eac44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:09.746416 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe6e1e-4f25-4f28-bcbc-2a49276eac44 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:09.747128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe6e1e-4f25-4f28-bcbc-2a49276eac44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:09.749095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe6e1e-4f25-4f28-bcbc-2a49276eac44 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:09.777576 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1269/2525] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:56:09 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:56:09.782244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd543a2a-de07-493b-980b-d2c9e405e598 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:09.792396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd543a2a-de07-493b-980b-d2c9e405e598 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:09.849545 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651b90b-f2fe-44ff-968b-e3ca22698300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:09.850224 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651b90b-f2fe-44ff-968b-e3ca22698300 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:09.850509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651b90b-f2fe-44ff-968b-e3ca22698300 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:09.850785 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651b90b-f2fe-44ff-968b-e3ca22698300 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:09.851478 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651b90b-f2fe-44ff-968b-e3ca22698300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:09.853469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651b90b-f2fe-44ff-968b-e3ca22698300 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:09.875990 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1257/2526] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:56:09 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:56:09.880164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe6e1e-4f25-4f28-bcbc-2a49276eac44 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:09.891444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe6e1e-4f25-4f28-bcbc-2a49276eac44 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:09.896619 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919645aa-02d4-4684-ac91-de54caf68ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:09.897232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919645aa-02d4-4684-ac91-de54caf68ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:09.897514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919645aa-02d4-4684-ac91-de54caf68ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:09.897790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919645aa-02d4-4684-ac91-de54caf68ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:09.898551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919645aa-02d4-4684-ac91-de54caf68ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:09.900125 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919645aa-02d4-4684-ac91-de54caf68ba9 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:09.914022 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1270/2527] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:56:09.922662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651b90b-f2fe-44ff-968b-e3ca22698300 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:09.936190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651b90b-f2fe-44ff-968b-e3ca22698300 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:09.941280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fba88-20c0-41bd-b024-d5b7fbde02be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:09.941969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fba88-20c0-41bd-b024-d5b7fbde02be admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:09.942253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fba88-20c0-41bd-b024-d5b7fbde02be admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:09.942531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fba88-20c0-41bd-b024-d5b7fbde02be admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:09.943271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fba88-20c0-41bd-b024-d5b7fbde02be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:09.944921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fba88-20c0-41bd-b024-d5b7fbde02be admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:09.957093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1258/2528] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:56:09.961369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919645aa-02d4-4684-ac91-de54caf68ba9 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:09.974117 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919645aa-02d4-4684-ac91-de54caf68ba9 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:09.979038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f90b4-214c-46ac-bcaf-0b8b3c86b4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:09.979943 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f90b4-214c-46ac-bcaf-0b8b3c86b4f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:09.980222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f90b4-214c-46ac-bcaf-0b8b3c86b4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:09.980498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f90b4-214c-46ac-bcaf-0b8b3c86b4f9 admin admin] PATH_INFO: `/v3/projects/30e986dad120481a8e2896d79e8fa6e4/users/76225062f589463c939a817e6abbc61e/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:10.004260 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f90b4-214c-46ac-bcaf-0b8b3c86b4f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30e986dad120481a8e2896d79e8fa6e4, user_id=76225062f589463c939a817e6abbc61e, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:10.007058 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f90b4-214c-46ac-bcaf-0b8b3c86b4f9 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:10.024476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1271/2529] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:56:09 2022] PUT /identity/v3/projects/30e986dad120481a8e2896d79e8fa6e4/users/76225062f589463c939a817e6abbc61e/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:56:10.035533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba62896-9836-4871-ae7f-1c90eaad086e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:10.035881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba62896-9836-4871-ae7f-1c90eaad086e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:10.036194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba62896-9836-4871-ae7f-1c90eaad086e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:10.064661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5ba62896-9836-4871-ae7f-1c90eaad086e None None] MFA Rules not processed for user `76225062f589463c939a817e6abbc61e`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:56:10.067421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5ba62896-9836-4871-ae7f-1c90eaad086e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:10.130675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1259/2530] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:56:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:56:10.142520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f90b4-214c-46ac-bcaf-0b8b3c86b4f9 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:10.144236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-485f90b4-214c-46ac-bcaf-0b8b3c86b4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:10.196733 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:56:10.203517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f90b4-214c-46ac-bcaf-0b8b3c86b4f9 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:10.208803 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd34c-a8b5-4616-bd07-4cf5b08af2f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:10.209579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd34c-a8b5-4616-bd07-4cf5b08af2f6 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:10.209969 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd34c-a8b5-4616-bd07-4cf5b08af2f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:10.210406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd34c-a8b5-4616-bd07-4cf5b08af2f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:10.220436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddd34c-a8b5-4616-bd07-4cf5b08af2f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:10.221913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddd34c-a8b5-4616-bd07-4cf5b08af2f6 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:10.231524 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1272/2531] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:56:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:56:20.127702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:56:20.227197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:56:48.723316 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba62896-9836-4871-ae7f-1c90eaad086e None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:48.724811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5ba62896-9836-4871-ae7f-1c90eaad086e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:48.786975 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba62896-9836-4871-ae7f-1c90eaad086e None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:48.790645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45634cb4-adb0-4f94-a1cb-06fc4f3a889a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:48.791300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45634cb4-adb0-4f94-a1cb-06fc4f3a889a service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:48.791588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45634cb4-adb0-4f94-a1cb-06fc4f3a889a service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:48.791869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45634cb4-adb0-4f94-a1cb-06fc4f3a889a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:48.794070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-45634cb4-adb0-4f94-a1cb-06fc4f3a889a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:48.805262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45634cb4-adb0-4f94-a1cb-06fc4f3a889a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:48.806706 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45634cb4-adb0-4f94-a1cb-06fc4f3a889a service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:48.816164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1260/2532] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:56:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:56:57.089112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd34c-a8b5-4616-bd07-4cf5b08af2f6 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:57.155372 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd34c-a8b5-4616-bd07-4cf5b08af2f6 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:57.160564 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:57.161764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:57.162061 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:57.162352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] PATH_INFO: `/v3/users/76225062f589463c939a817e6abbc61e` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:57.163864 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] RBAC: Authorizing `identity:delete_user(user_id=76225062f589463c939a817e6abbc61e)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:57.165851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:57.219996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76225062f589463c939a817e6abbc61e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:57.220702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76225062f589463c939a817e6abbc61e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:57.232109 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76225062f589463c939a817e6abbc61e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:57.251238 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76225062f589463c939a817e6abbc61e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:57.260546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76225062f589463c939a817e6abbc61e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:57.267801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1273/2533] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:56:57 2022] DELETE /identity/v3/users/76225062f589463c939a817e6abbc61e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:56:57.272452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2e3fe-6be1-4e89-a01e-0bc9a22c9809 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:57.272976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2e3fe-6be1-4e89-a01e-0bc9a22c9809 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:57.273408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2e3fe-6be1-4e89-a01e-0bc9a22c9809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:57.299234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ffa2e3fe-6be1-4e89-a01e-0bc9a22c9809 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:56:57.303642 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ffa2e3fe-6be1-4e89-a01e-0bc9a22c9809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:57.377836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1261/2534] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:56:57.387831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:57.393007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:57.471504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f223d3-2072-4ffa-8724-8fd8736f9eba admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:57.476534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2247f-cd0f-4c90-a256-9306e6179094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:57.477377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2247f-cd0f-4c90-a256-9306e6179094 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:57.477667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2247f-cd0f-4c90-a256-9306e6179094 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:57.477966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2247f-cd0f-4c90-a256-9306e6179094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:57.490854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2247f-cd0f-4c90-a256-9306e6179094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:57.492492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2247f-cd0f-4c90-a256-9306e6179094 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:57.504441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1274/2535] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:56:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:56:57.819235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2e3fe-6be1-4e89-a01e-0bc9a22c9809 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:57.822559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ffa2e3fe-6be1-4e89-a01e-0bc9a22c9809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:57.841813 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2e3fe-6be1-4e89-a01e-0bc9a22c9809 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:57.856606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeccb79-f2a4-4da8-b921-f80742367671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:57.858059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeccb79-f2a4-4da8-b921-f80742367671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:57.858523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeccb79-f2a4-4da8-b921-f80742367671 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:57.858976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeccb79-f2a4-4da8-b921-f80742367671 admin admin] PATH_INFO: `/v3/projects/30e986dad120481a8e2896d79e8fa6e4` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:57.861720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeccb79-f2a4-4da8-b921-f80742367671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30e986dad120481a8e2896d79e8fa6e4)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:57.865168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeccb79-f2a4-4da8-b921-f80742367671 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:57.934989 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4eeccb79-f2a4-4da8-b921-f80742367671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30e986dad120481a8e2896d79e8fa6e4', 'request_id': 'req-4eeccb79-f2a4-4da8-b921-f80742367671'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:57.944164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4eeccb79-f2a4-4da8-b921-f80742367671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30e986dad120481a8e2896d79e8fa6e4', 'request_id': 'req-4eeccb79-f2a4-4da8-b921-f80742367671'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:57.951555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4eeccb79-f2a4-4da8-b921-f80742367671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30e986dad120481a8e2896d79e8fa6e4', 'request_id': 'req-4eeccb79-f2a4-4da8-b921-f80742367671'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:57.963506 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4eeccb79-f2a4-4da8-b921-f80742367671 admin admin] The token cache is being invalidate because project 30e986dad120481a8e2896d79e8fa6e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:56:57.963801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4eeccb79-f2a4-4da8-b921-f80742367671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:57.964473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4eeccb79-f2a4-4da8-b921-f80742367671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:56:57.966341 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1262/2536] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:56:57 2022] DELETE /identity/v3/projects/30e986dad120481a8e2896d79e8fa6e4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:56:57.971495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6796d-f82c-4c04-a786-b1c8b077fb08 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:57.971793 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6796d-f82c-4c04-a786-b1c8b077fb08 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:57.972072 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6796d-f82c-4c04-a786-b1c8b077fb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:58.018845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a8c6796d-f82c-4c04-a786-b1c8b077fb08 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:56:58.020497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a8c6796d-f82c-4c04-a786-b1c8b077fb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:58.039273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:56:58.085351 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1275/2537] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:56:58.093421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e11c3c-9178-4cc2-b7d2-8a832e488607 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:58.093796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e11c3c-9178-4cc2-b7d2-8a832e488607 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:58.094077 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e11c3c-9178-4cc2-b7d2-8a832e488607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:58.113902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-e7e11c3c-9178-4cc2-b7d2-8a832e488607 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:56:58.115108 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e7e11c3c-9178-4cc2-b7d2-8a832e488607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:58.127677 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1263/2538] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:56:58.132070 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6796d-f82c-4c04-a786-b1c8b077fb08 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:58.142128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6796d-f82c-4c04-a786-b1c8b077fb08 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:58.147241 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab277e49-9257-4eb1-bc21-0b9edeb42cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:58.147915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab277e49-9257-4eb1-bc21-0b9edeb42cff admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:58.148194 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab277e49-9257-4eb1-bc21-0b9edeb42cff admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:58.148466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab277e49-9257-4eb1-bc21-0b9edeb42cff admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:58.149246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab277e49-9257-4eb1-bc21-0b9edeb42cff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:56:58.149549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab277e49-9257-4eb1-bc21-0b9edeb42cff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:58.151159 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab277e49-9257-4eb1-bc21-0b9edeb42cff admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:58.164455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1276/2539] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:56:58.168945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e11c3c-9178-4cc2-b7d2-8a832e488607 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:58.179445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e11c3c-9178-4cc2-b7d2-8a832e488607 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:58.184088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15573e1d-112b-417f-8c50-55f63cd83cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:58.184782 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15573e1d-112b-417f-8c50-55f63cd83cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:58.185084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15573e1d-112b-417f-8c50-55f63cd83cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:58.185360 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15573e1d-112b-417f-8c50-55f63cd83cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:58.186119 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15573e1d-112b-417f-8c50-55f63cd83cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:58.188562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15573e1d-112b-417f-8c50-55f63cd83cd1 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:58.211446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:56:58.225585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1264/2540] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:56:58 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:56:58.226755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab277e49-9257-4eb1-bc21-0b9edeb42cff admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:58.238085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab277e49-9257-4eb1-bc21-0b9edeb42cff admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:58.297267 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3059c-a6b1-4134-a383-da637101d997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:58.298466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3059c-a6b1-4134-a383-da637101d997 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:58.298756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3059c-a6b1-4134-a383-da637101d997 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:58.299038 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3059c-a6b1-4134-a383-da637101d997 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:58.299764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3059c-a6b1-4134-a383-da637101d997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:58.301741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3059c-a6b1-4134-a383-da637101d997 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:58.324127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1277/2541] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:56:58 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:56:58.330914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15573e1d-112b-417f-8c50-55f63cd83cd1 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:58.341547 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15573e1d-112b-417f-8c50-55f63cd83cd1 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:58.346943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f46779-74d8-4a51-98a8-3b6a18feabc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:58.347622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f46779-74d8-4a51-98a8-3b6a18feabc1 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:58.347905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f46779-74d8-4a51-98a8-3b6a18feabc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:58.348187 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f46779-74d8-4a51-98a8-3b6a18feabc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:58.348970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f46779-74d8-4a51-98a8-3b6a18feabc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:58.350584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f46779-74d8-4a51-98a8-3b6a18feabc1 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:58.362897 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1265/2542] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:56:58.366927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3059c-a6b1-4134-a383-da637101d997 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:58.377421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3059c-a6b1-4134-a383-da637101d997 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:58.382062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658b0b4-ece2-40c7-ae5f-41eb996c0aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:58.382739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658b0b4-ece2-40c7-ae5f-41eb996c0aed admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:58.383013 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658b0b4-ece2-40c7-ae5f-41eb996c0aed admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:58.383293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658b0b4-ece2-40c7-ae5f-41eb996c0aed admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:58.384022 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658b0b4-ece2-40c7-ae5f-41eb996c0aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:58.385655 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658b0b4-ece2-40c7-ae5f-41eb996c0aed admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:58.396832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1278/2543] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:56:58.400922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f46779-74d8-4a51-98a8-3b6a18feabc1 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:58.411660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f46779-74d8-4a51-98a8-3b6a18feabc1 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:58.416396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3a14f-210b-465b-9951-56303c7036b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:58.417343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3a14f-210b-465b-9951-56303c7036b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:58.417628 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3a14f-210b-465b-9951-56303c7036b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:58.417907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3a14f-210b-465b-9951-56303c7036b2 admin admin] PATH_INFO: `/v3/projects/5005a107ff1940908069356c788185f6/users/a1a3d224feb94a1baa75a03bc8d1132a/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:58.435303 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3a14f-210b-465b-9951-56303c7036b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5005a107ff1940908069356c788185f6, user_id=a1a3d224feb94a1baa75a03bc8d1132a, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:58.437953 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3a14f-210b-465b-9951-56303c7036b2 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:58.452045 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1266/2544] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:56:58 2022] PUT /identity/v3/projects/5005a107ff1940908069356c788185f6/users/a1a3d224feb94a1baa75a03bc8d1132a/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:56:58.457374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaa914-0dd0-47f1-b9e1-65a27aac488d None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:58.457669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaa914-0dd0-47f1-b9e1-65a27aac488d None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:58.457978 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaa914-0dd0-47f1-b9e1-65a27aac488d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:58.478055 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a3eaa914-0dd0-47f1-b9e1-65a27aac488d None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:56:58.479603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a3eaa914-0dd0-47f1-b9e1-65a27aac488d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:58.553724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1279/2545] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:56:58.562248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3a14f-210b-465b-9951-56303c7036b2 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:56:58.564090 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-57f3a14f-210b-465b-9951-56303c7036b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:56:58.627715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3a14f-210b-465b-9951-56303c7036b2 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:56:58.631423 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972314e-d5c4-4baf-bae6-d383dcb397a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:56:58.632107 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972314e-d5c4-4baf-bae6-d383dcb397a4 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:56:58.632401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972314e-d5c4-4baf-bae6-d383dcb397a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:56:58.632716 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972314e-d5c4-4baf-bae6-d383dcb397a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:56:58.643136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972314e-d5c4-4baf-bae6-d383dcb397a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:56:58.644584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972314e-d5c4-4baf-bae6-d383dcb397a4 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:56:58.654499 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1267/2546] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:57:03.064767 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4ec20-3663-4d7e-9040-672d7ab0a08b None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:03.065354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4ec20-3663-4d7e-9040-672d7ab0a08b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:03.065477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4ec20-3663-4d7e-9040-672d7ab0a08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:03.102611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-aad4ec20-3663-4d7e-9040-672d7ab0a08b None None] MFA Rules not processed for user `a1a3d224feb94a1baa75a03bc8d1132a`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:03.104244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-aad4ec20-3663-4d7e-9040-672d7ab0a08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:03.127327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:03.236625 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1280/2547] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:03.238510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972314e-d5c4-4baf-bae6-d383dcb397a4 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:03.254248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972314e-d5c4-4baf-bae6-d383dcb397a4 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:03.265913 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7c319-9189-4d7c-8c90-e6ad25b87f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:03.266693 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7c319-9189-4d7c-8c90-e6ad25b87f3c admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:03.266979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7c319-9189-4d7c-8c90-e6ad25b87f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:03.267257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7c319-9189-4d7c-8c90-e6ad25b87f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:03.267969 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7c319-9189-4d7c-8c90-e6ad25b87f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:03.269966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7c319-9189-4d7c-8c90-e6ad25b87f3c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:03.309091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1268/2548] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:57:03 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:57:03.312923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4ec20-3663-4d7e-9040-672d7ab0a08b None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:03.325631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4ec20-3663-4d7e-9040-672d7ab0a08b None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:03.405972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd55f2-7058-43bf-9186-d4d078143561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:03.407253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd55f2-7058-43bf-9186-d4d078143561 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:03.407545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd55f2-7058-43bf-9186-d4d078143561 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:03.407827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd55f2-7058-43bf-9186-d4d078143561 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:03.408546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd55f2-7058-43bf-9186-d4d078143561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:03.410575 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd55f2-7058-43bf-9186-d4d078143561 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:03.438090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1281/2549] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:57:03 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:57:03.441998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7c319-9189-4d7c-8c90-e6ad25b87f3c admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:03.462322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7c319-9189-4d7c-8c90-e6ad25b87f3c admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:03.468148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fdb1e-32b0-4baf-b719-1e5fa6d688aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:03.468918 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fdb1e-32b0-4baf-b719-1e5fa6d688aa admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:03.469200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fdb1e-32b0-4baf-b719-1e5fa6d688aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:03.469494 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fdb1e-32b0-4baf-b719-1e5fa6d688aa admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:03.470956 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fdb1e-32b0-4baf-b719-1e5fa6d688aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:03.471883 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fdb1e-32b0-4baf-b719-1e5fa6d688aa admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:03.484559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1269/2550] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:57:03.488806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd55f2-7058-43bf-9186-d4d078143561 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:03.501434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd55f2-7058-43bf-9186-d4d078143561 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:03.506963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2b777-9a35-4031-9fb5-544ae0c80294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:03.507720 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2b777-9a35-4031-9fb5-544ae0c80294 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:03.508164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2b777-9a35-4031-9fb5-544ae0c80294 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:03.508447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2b777-9a35-4031-9fb5-544ae0c80294 admin admin] PATH_INFO: `/v3/projects/c9b5e4cfcdee4d96a8b9f08f8ed323a1/users/c4f250476ffc4958aafa960ef85d6546/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:03.530940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f2b777-9a35-4031-9fb5-544ae0c80294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b5e4cfcdee4d96a8b9f08f8ed323a1, user_id=c4f250476ffc4958aafa960ef85d6546, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:03.533945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f2b777-9a35-4031-9fb5-544ae0c80294 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:03.554679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1282/2551] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:57:03 2022] PUT /identity/v3/projects/c9b5e4cfcdee4d96a8b9f08f8ed323a1/users/c4f250476ffc4958aafa960ef85d6546/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:03.560073 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fdb1e-32b0-4baf-b719-1e5fa6d688aa admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:03.571789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fdb1e-32b0-4baf-b719-1e5fa6d688aa admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:03.652819 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc69e17-938d-44c1-b9b7-aa1415ea82a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:03.653509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc69e17-938d-44c1-b9b7-aa1415ea82a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:03.653781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc69e17-938d-44c1-b9b7-aa1415ea82a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:03.654139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc69e17-938d-44c1-b9b7-aa1415ea82a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:03.654891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc69e17-938d-44c1-b9b7-aa1415ea82a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:03.656638 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc69e17-938d-44c1-b9b7-aa1415ea82a6 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:03.670487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1270/2552] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:57:03.675102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2b777-9a35-4031-9fb5-544ae0c80294 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:03.688539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2b777-9a35-4031-9fb5-544ae0c80294 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:03.694141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e62642-548b-4d40-b81d-1e6b3cadaf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:03.695318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e62642-548b-4d40-b81d-1e6b3cadaf73 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:03.695777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e62642-548b-4d40-b81d-1e6b3cadaf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:03.696231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e62642-548b-4d40-b81d-1e6b3cadaf73 admin admin] PATH_INFO: `/v3/projects/c9b5e4cfcdee4d96a8b9f08f8ed323a1/users/c4f250476ffc4958aafa960ef85d6546/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:03.698485 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e62642-548b-4d40-b81d-1e6b3cadaf73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b5e4cfcdee4d96a8b9f08f8ed323a1, user_id=c4f250476ffc4958aafa960ef85d6546, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:03.704039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e62642-548b-4d40-b81d-1e6b3cadaf73 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:03.714208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1283/2553] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:57:03 2022] PUT /identity/v3/projects/c9b5e4cfcdee4d96a8b9f08f8ed323a1/users/c4f250476ffc4958aafa960ef85d6546/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:05.327068 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc69e17-938d-44c1-b9b7-aa1415ea82a6 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:05.330101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-9dc69e17-938d-44c1-b9b7-aa1415ea82a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:05.370880 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:05.445987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc69e17-938d-44c1-b9b7-aa1415ea82a6 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:05.604250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc058fd-ba13-4f5a-a790-84118a821003 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:05.605141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc058fd-ba13-4f5a-a790-84118a821003 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:05.605899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc058fd-ba13-4f5a-a790-84118a821003 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:05.606469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc058fd-ba13-4f5a-a790-84118a821003 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:05.612158 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7bc058fd-ba13-4f5a-a790-84118a821003 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:05.654974 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc058fd-ba13-4f5a-a790-84118a821003 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:05.660367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc058fd-ba13-4f5a-a790-84118a821003 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:05.693987 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1271/2554] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:57:08.841800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fa7a8-4e37-4d80-8429-f3954616e094 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:08.842306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fa7a8-4e37-4d80-8429-f3954616e094 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:08.842501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fa7a8-4e37-4d80-8429-f3954616e094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:08.893641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-039fa7a8-4e37-4d80-8429-f3954616e094 None None] MFA Rules not processed for user `c4f250476ffc4958aafa960ef85d6546`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:08.894898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-039fa7a8-4e37-4d80-8429-f3954616e094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:08.994177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:09.023923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1284/2555] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:09.031277 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc058fd-ba13-4f5a-a790-84118a821003 service nova] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:09.045948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc058fd-ba13-4f5a-a790-84118a821003 service nova] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:09.052900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a518f82-212b-4814-baa3-fc3eae46610a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:09.053563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a518f82-212b-4814-baa3-fc3eae46610a service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:09.053853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a518f82-212b-4814-baa3-fc3eae46610a service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:09.054135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a518f82-212b-4814-baa3-fc3eae46610a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:09.120462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a518f82-212b-4814-baa3-fc3eae46610a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:09.122130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a518f82-212b-4814-baa3-fc3eae46610a service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:09.132003 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1272/2556] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:57:10.586530 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cab46-94f4-4db1-8743-d2c1536fcff6 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:10.587016 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cab46-94f4-4db1-8743-d2c1536fcff6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:10.587136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cab46-94f4-4db1-8743-d2c1536fcff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:10.647996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a19cab46-94f4-4db1-8743-d2c1536fcff6 None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:10.649319 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a19cab46-94f4-4db1-8743-d2c1536fcff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:10.743294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1285/2557] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:57:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:10.807562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a518f82-212b-4814-baa3-fc3eae46610a service nova] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:10.809198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3a518f82-212b-4814-baa3-fc3eae46610a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:10.878129 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:10.924251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a518f82-212b-4814-baa3-fc3eae46610a service nova] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:10.933243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dee11-b86a-4258-852d-e64359228c7a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:10.933985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dee11-b86a-4258-852d-e64359228c7a service swift] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:10.934271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dee11-b86a-4258-852d-e64359228c7a service swift] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:10.934604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dee11-b86a-4258-852d-e64359228c7a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:10.944943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dee11-b86a-4258-852d-e64359228c7a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:10.946553 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dee11-b86a-4258-852d-e64359228c7a service swift] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:10.957846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1273/2558] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:57:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:57:17.179107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cab46-94f4-4db1-8743-d2c1536fcff6 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:17.199917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cab46-94f4-4db1-8743-d2c1536fcff6 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:17.207085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c46fe1-8eaa-4876-91ba-5f6bb265be91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:17.207737 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c46fe1-8eaa-4876-91ba-5f6bb265be91 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:17.208056 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c46fe1-8eaa-4876-91ba-5f6bb265be91 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:17.208306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c46fe1-8eaa-4876-91ba-5f6bb265be91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:17.210214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-04c46fe1-8eaa-4876-91ba-5f6bb265be91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:17.223436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c46fe1-8eaa-4876-91ba-5f6bb265be91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:17.228167 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c46fe1-8eaa-4876-91ba-5f6bb265be91 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:17.253666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1286/2559] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:57:20.952547 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=18 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:23.050678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dee11-b86a-4258-852d-e64359228c7a service swift] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:23.129573 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dee11-b86a-4258-852d-e64359228c7a service swift] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:23.134574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdaaa6e-dd8b-485c-a0da-9c40cdad55e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:23.135529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaaa6e-dd8b-485c-a0da-9c40cdad55e7 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:23.136006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaaa6e-dd8b-485c-a0da-9c40cdad55e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:23.136490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaaa6e-dd8b-485c-a0da-9c40cdad55e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:23.139019 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-afdaaa6e-dd8b-485c-a0da-9c40cdad55e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:23.220077 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdaaa6e-dd8b-485c-a0da-9c40cdad55e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:23.221572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdaaa6e-dd8b-485c-a0da-9c40cdad55e7 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:23.232435 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1274/2560] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:57:27.161503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c46fe1-8eaa-4876-91ba-5f6bb265be91 service nova] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:27.163854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-04c46fe1-8eaa-4876-91ba-5f6bb265be91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:27.185719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c46fe1-8eaa-4876-91ba-5f6bb265be91 service nova] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:27.190991 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151625d-ba1a-40c0-ab08-912b2954b2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:27.192013 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151625d-ba1a-40c0-ab08-912b2954b2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:27.192013 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151625d-ba1a-40c0-ab08-912b2954b2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:27.193350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151625d-ba1a-40c0-ab08-912b2954b2a5 admin admin] PATH_INFO: `/v3/users/718ca333456f4de5b98f191ab40cca0b` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:27.208979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151625d-ba1a-40c0-ab08-912b2954b2a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=718ca333456f4de5b98f191ab40cca0b)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:27.210979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151625d-ba1a-40c0-ab08-912b2954b2a5 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:27.267895 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7151625d-ba1a-40c0-ab08-912b2954b2a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '718ca333456f4de5b98f191ab40cca0b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:27.268588 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7151625d-ba1a-40c0-ab08-912b2954b2a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '718ca333456f4de5b98f191ab40cca0b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:27.277302 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7151625d-ba1a-40c0-ab08-912b2954b2a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '718ca333456f4de5b98f191ab40cca0b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:27.294147 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7151625d-ba1a-40c0-ab08-912b2954b2a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '718ca333456f4de5b98f191ab40cca0b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:27.302857 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7151625d-ba1a-40c0-ab08-912b2954b2a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '718ca333456f4de5b98f191ab40cca0b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:27.309234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1287/2561] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:57:27 2022] DELETE /identity/v3/users/718ca333456f4de5b98f191ab40cca0b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:27.512485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdaaa6e-dd8b-485c-a0da-9c40cdad55e7 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:27.527125 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdaaa6e-dd8b-485c-a0da-9c40cdad55e7 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:27.587669 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:27.588537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:27.588929 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:27.589137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f admin admin] PATH_INFO: `/v3/projects/9313fe06660f4049be47f901c3fc6f76` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:27.600940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9313fe06660f4049be47f901c3fc6f76)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:27.602879 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:27.650580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:27.654898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9313fe06660f4049be47f901c3fc6f76', 'request_id': 'req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:27.664024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9313fe06660f4049be47f901c3fc6f76', 'request_id': 'req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:27.669631 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9313fe06660f4049be47f901c3fc6f76', 'request_id': 'req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:27.681836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f admin admin] The token cache is being invalidate because project 9313fe06660f4049be47f901c3fc6f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:57:27.682133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:27.682812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-6c8759fd-5044-4dbb-8d7e-fedf2d09e51f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:27.684476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1275/2562] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:57:27 2022] DELETE /identity/v3/projects/9313fe06660f4049be47f901c3fc6f76 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:27.690457 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d770a2-12aa-430f-8eda-7432689d9e88 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:27.690744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d770a2-12aa-430f-8eda-7432689d9e88 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:27.691028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d770a2-12aa-430f-8eda-7432689d9e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:27.718971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d9d770a2-12aa-430f-8eda-7432689d9e88 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:27.720815 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d9d770a2-12aa-430f-8eda-7432689d9e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:27.735863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:27.799351 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1288/2563] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:27.807625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119716fe-7fbd-4780-89c1-5b8ea3181812 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:27.807933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119716fe-7fbd-4780-89c1-5b8ea3181812 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:27.808367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119716fe-7fbd-4780-89c1-5b8ea3181812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:27.831169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-119716fe-7fbd-4780-89c1-5b8ea3181812 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:27.832445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-119716fe-7fbd-4780-89c1-5b8ea3181812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:27.845454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1276/2564] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:27.849722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d770a2-12aa-430f-8eda-7432689d9e88 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:27.861039 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d770a2-12aa-430f-8eda-7432689d9e88 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:27.865832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7f40b-3da8-49df-9476-a4c73d63c226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:27.866636 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7f40b-3da8-49df-9476-a4c73d63c226 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:27.866923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7f40b-3da8-49df-9476-a4c73d63c226 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:27.867197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7f40b-3da8-49df-9476-a4c73d63c226 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:27.867932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7f40b-3da8-49df-9476-a4c73d63c226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:57:27.868236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7f40b-3da8-49df-9476-a4c73d63c226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:27.870195 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7f40b-3da8-49df-9476-a4c73d63c226 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:27.883974 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1289/2565] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:57:27.888652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119716fe-7fbd-4780-89c1-5b8ea3181812 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:27.899898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119716fe-7fbd-4780-89c1-5b8ea3181812 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:27.905661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9779f-6f7f-4172-8850-e2ea1f86f0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:27.906348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9779f-6f7f-4172-8850-e2ea1f86f0c3 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:27.906625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9779f-6f7f-4172-8850-e2ea1f86f0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:27.906899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9779f-6f7f-4172-8850-e2ea1f86f0c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:27.907608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9779f-6f7f-4172-8850-e2ea1f86f0c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:27.909530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9779f-6f7f-4172-8850-e2ea1f86f0c3 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:27.941712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1277/2566] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:57:27 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:57:27.945905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7f40b-3da8-49df-9476-a4c73d63c226 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:27.956177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7f40b-3da8-49df-9476-a4c73d63c226 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:28.016820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c864ac-c553-4b96-8016-5818e9354596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:28.017523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c864ac-c553-4b96-8016-5818e9354596 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.017806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c864ac-c553-4b96-8016-5818e9354596 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.018085 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c864ac-c553-4b96-8016-5818e9354596 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.018808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c864ac-c553-4b96-8016-5818e9354596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:28.020779 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c864ac-c553-4b96-8016-5818e9354596 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:28.043073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1290/2567] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:57:27 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:57:28.047779 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9779f-6f7f-4172-8850-e2ea1f86f0c3 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:28.058527 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9779f-6f7f-4172-8850-e2ea1f86f0c3 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:28.063943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414a477-0208-4388-b31d-9cf08517d7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:28.064640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414a477-0208-4388-b31d-9cf08517d7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.064926 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414a477-0208-4388-b31d-9cf08517d7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.065202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414a477-0208-4388-b31d-9cf08517d7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.065945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414a477-0208-4388-b31d-9cf08517d7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:28.067537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414a477-0208-4388-b31d-9cf08517d7f0 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:28.078505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1278/2568] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:57:28.082755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c864ac-c553-4b96-8016-5818e9354596 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:28.093062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c864ac-c553-4b96-8016-5818e9354596 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:28.098271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea823cf-2c97-4557-8988-c567e7b93218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:28.098941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea823cf-2c97-4557-8988-c567e7b93218 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.099227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea823cf-2c97-4557-8988-c567e7b93218 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.099508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea823cf-2c97-4557-8988-c567e7b93218 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.100244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea823cf-2c97-4557-8988-c567e7b93218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:28.101892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea823cf-2c97-4557-8988-c567e7b93218 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:28.113698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1291/2569] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:57:28.118280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414a477-0208-4388-b31d-9cf08517d7f0 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:28.129385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414a477-0208-4388-b31d-9cf08517d7f0 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:28.134189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1a0e4-6073-4cc1-a061-6ca87ab497ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:28.135081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1a0e4-6073-4cc1-a061-6ca87ab497ed admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.135360 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1a0e4-6073-4cc1-a061-6ca87ab497ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.135633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1a0e4-6073-4cc1-a061-6ca87ab497ed admin admin] PATH_INFO: `/v3/projects/b2adde584b004ae6b648453c5477bace/users/5ea28d3129c74588b2c108e339d918ec/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.150989 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1a0e4-6073-4cc1-a061-6ca87ab497ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2adde584b004ae6b648453c5477bace, user_id=5ea28d3129c74588b2c108e339d918ec, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:28.153659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1a0e4-6073-4cc1-a061-6ca87ab497ed admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:28.168556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1279/2570] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:57:28 2022] PUT /identity/v3/projects/b2adde584b004ae6b648453c5477bace/users/5ea28d3129c74588b2c108e339d918ec/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:28.183212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51601a2-1414-49d4-8a18-511122ed937f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.183791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51601a2-1414-49d4-8a18-511122ed937f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.183791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51601a2-1414-49d4-8a18-511122ed937f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.215353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d51601a2-1414-49d4-8a18-511122ed937f None None] MFA Rules not processed for user `5ea28d3129c74588b2c108e339d918ec`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:28.216509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d51601a2-1414-49d4-8a18-511122ed937f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:28.298981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1292/2571] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:28.303176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1a0e4-6073-4cc1-a061-6ca87ab497ed admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:28.373697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1a0e4-6073-4cc1-a061-6ca87ab497ed admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:28.378363 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c545feb-298c-458b-9f85-f4b88b82111c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:28.379022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c545feb-298c-458b-9f85-f4b88b82111c admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.379306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c545feb-298c-458b-9f85-f4b88b82111c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.379583 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c545feb-298c-458b-9f85-f4b88b82111c admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.380295 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c545feb-298c-458b-9f85-f4b88b82111c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:28.382214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c545feb-298c-458b-9f85-f4b88b82111c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:28.411042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1280/2572] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:57:28 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:57:28.415287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51601a2-1414-49d4-8a18-511122ed937f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:28.428488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51601a2-1414-49d4-8a18-511122ed937f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:28.437288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:28.494785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd4554-9e07-4b97-82e8-ec08a5a53d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:28.495467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd4554-9e07-4b97-82e8-ec08a5a53d6f admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.495749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd4554-9e07-4b97-82e8-ec08a5a53d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.496027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd4554-9e07-4b97-82e8-ec08a5a53d6f admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.496773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd4554-9e07-4b97-82e8-ec08a5a53d6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:28.498704 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd4554-9e07-4b97-82e8-ec08a5a53d6f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:28.530525 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1293/2573] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:57:28 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:57:28.532401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c545feb-298c-458b-9f85-f4b88b82111c admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:28.542912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c545feb-298c-458b-9f85-f4b88b82111c admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:28.548394 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcd01f-fb63-407f-80d5-cf78b76dc869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:28.549311 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcd01f-fb63-407f-80d5-cf78b76dc869 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.549596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcd01f-fb63-407f-80d5-cf78b76dc869 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.549884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcd01f-fb63-407f-80d5-cf78b76dc869 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.550604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcd01f-fb63-407f-80d5-cf78b76dc869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:28.552215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcd01f-fb63-407f-80d5-cf78b76dc869 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:28.563148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1281/2574] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:57:28.567456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd4554-9e07-4b97-82e8-ec08a5a53d6f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:28.577670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd4554-9e07-4b97-82e8-ec08a5a53d6f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:28.582896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f2405-a35c-41cd-a627-026fe1163429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:28.583778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f2405-a35c-41cd-a627-026fe1163429 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.584063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f2405-a35c-41cd-a627-026fe1163429 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.584342 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f2405-a35c-41cd-a627-026fe1163429 admin admin] PATH_INFO: `/v3/projects/14b5c60f9c7945bea96c2fa3bee5441a/users/160c770d29e743c392546b9ea0397da4/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.600963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f2405-a35c-41cd-a627-026fe1163429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b5c60f9c7945bea96c2fa3bee5441a, user_id=160c770d29e743c392546b9ea0397da4, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:28.603918 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f2405-a35c-41cd-a627-026fe1163429 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:28.617360 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1294/2575] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:57:28 2022] PUT /identity/v3/projects/14b5c60f9c7945bea96c2fa3bee5441a/users/160c770d29e743c392546b9ea0397da4/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:28.621661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcd01f-fb63-407f-80d5-cf78b76dc869 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:28.631744 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcd01f-fb63-407f-80d5-cf78b76dc869 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:28.672297 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:28.693748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef364d-636e-4b1f-aa5e-8e6a3169414b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:28.694411 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef364d-636e-4b1f-aa5e-8e6a3169414b admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.694704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef364d-636e-4b1f-aa5e-8e6a3169414b admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.694972 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef364d-636e-4b1f-aa5e-8e6a3169414b admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.695709 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef364d-636e-4b1f-aa5e-8e6a3169414b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:28.697348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef364d-636e-4b1f-aa5e-8e6a3169414b admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:28.709754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1282/2576] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:57:28.714171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f2405-a35c-41cd-a627-026fe1163429 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:28.725585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f2405-a35c-41cd-a627-026fe1163429 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:28.730296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628025b-c258-4451-85e6-f1aaf0aec479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:28.731187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628025b-c258-4451-85e6-f1aaf0aec479 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.731466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628025b-c258-4451-85e6-f1aaf0aec479 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.731739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628025b-c258-4451-85e6-f1aaf0aec479 admin admin] PATH_INFO: `/v3/projects/14b5c60f9c7945bea96c2fa3bee5441a/users/160c770d29e743c392546b9ea0397da4/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.733775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628025b-c258-4451-85e6-f1aaf0aec479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b5c60f9c7945bea96c2fa3bee5441a, user_id=160c770d29e743c392546b9ea0397da4, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:28.736418 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628025b-c258-4451-85e6-f1aaf0aec479 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:28.748005 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1295/2577] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:57:28 2022] PUT /identity/v3/projects/14b5c60f9c7945bea96c2fa3bee5441a/users/160c770d29e743c392546b9ea0397da4/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:28.758955 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7ad57-3b5b-4215-9bfa-70d0243d9e78 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.759344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7ad57-3b5b-4215-9bfa-70d0243d9e78 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.759698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7ad57-3b5b-4215-9bfa-70d0243d9e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.788815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-24d7ad57-3b5b-4215-9bfa-70d0243d9e78 None None] MFA Rules not processed for user `160c770d29e743c392546b9ea0397da4`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:28.790037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-24d7ad57-3b5b-4215-9bfa-70d0243d9e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:28.879091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1283/2578] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:28.891444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628025b-c258-4451-85e6-f1aaf0aec479 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:28.893593 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-0628025b-c258-4451-85e6-f1aaf0aec479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:28.967052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628025b-c258-4451-85e6-f1aaf0aec479 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:28.971656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe6564-4258-4dcf-ba9b-7104d033718c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:28.972453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe6564-4258-4dcf-ba9b-7104d033718c service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:28.973027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe6564-4258-4dcf-ba9b-7104d033718c service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:28.973563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe6564-4258-4dcf-ba9b-7104d033718c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:28.984365 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe6564-4258-4dcf-ba9b-7104d033718c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:28.985855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe6564-4258-4dcf-ba9b-7104d033718c service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:28.997113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1296/2579] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:57:29.094012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7ad57-3b5b-4215-9bfa-70d0243d9e78 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:29.104095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7ad57-3b5b-4215-9bfa-70d0243d9e78 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:29.107869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084400ee-efb6-4c48-ad83-168826363e68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:29.108766 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084400ee-efb6-4c48-ad83-168826363e68 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:29.109049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084400ee-efb6-4c48-ad83-168826363e68 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:29.111631 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084400ee-efb6-4c48-ad83-168826363e68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:29.176957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084400ee-efb6-4c48-ad83-168826363e68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:29.178402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084400ee-efb6-4c48-ad83-168826363e68 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:29.188145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1284/2580] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:57:34.568817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a126b3-a6d1-4e53-b677-cdc7340b5d57 None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:34.569379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a126b3-a6d1-4e53-b677-cdc7340b5d57 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:34.569547 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a126b3-a6d1-4e53-b677-cdc7340b5d57 None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:34.571098 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1297/2581] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:57:34.579119 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41870ec-c281-465b-b1e1-7eaaec4fd326 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:34.579647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41870ec-c281-465b-b1e1-7eaaec4fd326 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:34.579910 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41870ec-c281-465b-b1e1-7eaaec4fd326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:34.623768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-d41870ec-c281-465b-b1e1-7eaaec4fd326 None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:34.626238 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d41870ec-c281-465b-b1e1-7eaaec4fd326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:34.673036 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:34.697009 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1285/2582] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:57:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:34.707990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a126b3-a6d1-4e53-b677-cdc7340b5d57 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:34.709935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-81a126b3-a6d1-4e53-b677-cdc7340b5d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:34.770906 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a126b3-a6d1-4e53-b677-cdc7340b5d57 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:34.774476 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e9c0e-c111-4e9f-bf77-901c531862f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '925bf2da4df74e86aa84ec62a267d088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:34.775121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e9c0e-c111-4e9f-bf77-901c531862f0 service glance] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:34.775404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e9c0e-c111-4e9f-bf77-901c531862f0 service glance] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:34.775681 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e9c0e-c111-4e9f-bf77-901c531862f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:34.786303 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e9c0e-c111-4e9f-bf77-901c531862f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:34.787670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e9c0e-c111-4e9f-bf77-901c531862f0 service glance] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:34.799106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1298/2583] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:57:34.866355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243b13d-aff0-4434-94c1-ee4590dfda9a None None] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:34.866644 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243b13d-aff0-4434-94c1-ee4590dfda9a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:34.866924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243b13d-aff0-4434-94c1-ee4590dfda9a None None] PATH_INFO: `/` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:34.868356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1286/2584] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:57:34.873500 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea0abb-6d45-46aa-baea-8eabe5b922f1 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:34.873822 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea0abb-6d45-46aa-baea-8eabe5b922f1 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:34.874112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea0abb-6d45-46aa-baea-8eabe5b922f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:34.893128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:34.914920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-5aea0abb-6d45-46aa-baea-8eabe5b922f1 None None] MFA Rules not processed for user `62688bc3d8a449c5bc9e0d64cd2462b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:34.916152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-5aea0abb-6d45-46aa-baea-8eabe5b922f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:34.968323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1299/2585] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:57:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:34.980725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243b13d-aff0-4434-94c1-ee4590dfda9a None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:34.982468 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2243b13d-aff0-4434-94c1-ee4590dfda9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:34.996748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243b13d-aff0-4434-94c1-ee4590dfda9a None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:35.000257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736f876-2a34-4025-9db3-de14e67d94de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:35.001001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736f876-2a34-4025-9db3-de14e67d94de service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:35.001287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736f876-2a34-4025-9db3-de14e67d94de service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:35.001563 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736f876-2a34-4025-9db3-de14e67d94de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:35.009029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736f876-2a34-4025-9db3-de14e67d94de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:35.010466 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736f876-2a34-4025-9db3-de14e67d94de service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:35.018584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1287/2586] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:57:36.378227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854742f7-2e12-45cd-a460-68ec35708f8f None None] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:36.378793 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854742f7-2e12-45cd-a460-68ec35708f8f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:36.378793 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854742f7-2e12-45cd-a460-68ec35708f8f None None] PATH_INFO: `/` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:36.380185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1300/2587] 10.222.0.52 () {54 vars in 921 bytes} [Mon Jul 25 08:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 08:57:36.386043 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eca4f9-cf3b-4a4b-bc01-ff687bd794c5 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:36.386596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eca4f9-cf3b-4a4b-bc01-ff687bd794c5 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:36.386684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eca4f9-cf3b-4a4b-bc01-ff687bd794c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:36.407898 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-44eca4f9-cf3b-4a4b-bc01-ff687bd794c5 None None] MFA Rules not processed for user `8b476d6b82ae464b81fc8498dc0e68c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:36.409161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-44eca4f9-cf3b-4a4b-bc01-ff687bd794c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:36.419110 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1288/2588] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Jul 25 08:57:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:36.431604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854742f7-2e12-45cd-a460-68ec35708f8f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:36.443434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854742f7-2e12-45cd-a460-68ec35708f8f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:36.446981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2f487-54d5-4d72-907c-58edce126aaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:36.447621 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2f487-54d5-4d72-907c-58edce126aaa service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:36.447911 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2f487-54d5-4d72-907c-58edce126aaa service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:36.448189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2f487-54d5-4d72-907c-58edce126aaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:36.460791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2f487-54d5-4d72-907c-58edce126aaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:36.462529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2f487-54d5-4d72-907c-58edce126aaa service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:36.475829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1301/2589] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:57:36.542370 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eca4f9-cf3b-4a4b-bc01-ff687bd794c5 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:36.543954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-44eca4f9-cf3b-4a4b-bc01-ff687bd794c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:36.617413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eca4f9-cf3b-4a4b-bc01-ff687bd794c5 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:36.622757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d60619-ad20-4a26-a778-1cd3eb610699 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:36.623408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d60619-ad20-4a26-a778-1cd3eb610699 service placement] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:36.623691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d60619-ad20-4a26-a778-1cd3eb610699 service placement] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:36.623972 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d60619-ad20-4a26-a778-1cd3eb610699 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:36.625797 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-70d60619-ad20-4a26-a778-1cd3eb610699 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:36.637599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d60619-ad20-4a26-a778-1cd3eb610699 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:36.639370 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d60619-ad20-4a26-a778-1cd3eb610699 service placement] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:36.649768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1289/2590] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:57:41.083752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ac335-7c69-4d62-988f-e0014fb283a1 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:41.084341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ac335-7c69-4d62-988f-e0014fb283a1 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:41.084341 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ac335-7c69-4d62-988f-e0014fb283a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:41.119603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-286ac335-7c69-4d62-988f-e0014fb283a1 None None] MFA Rules not processed for user `a1a3d224feb94a1baa75a03bc8d1132a`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:41.121735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-286ac335-7c69-4d62-988f-e0014fb283a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:41.190531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1302/2591] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:57:41.491194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb214d2-5314-4f36-b79b-2765bd0814be None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:41.491727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb214d2-5314-4f36-b79b-2765bd0814be None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:41.491727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb214d2-5314-4f36-b79b-2765bd0814be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:41.513246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-8fb214d2-5314-4f36-b79b-2765bd0814be None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:41.514928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8fb214d2-5314-4f36-b79b-2765bd0814be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:41.581435 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1290/2592] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Jul 25 08:57:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:57:41.597183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ac335-7c69-4d62-988f-e0014fb283a1 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:41.598897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-286ac335-7c69-4d62-988f-e0014fb283a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:41.685051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ac335-7c69-4d62-988f-e0014fb283a1 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:41.692323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bdd9f-c303-4986-986d-bdfc58e35484 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:41.692323 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bdd9f-c303-4986-986d-bdfc58e35484 service swift] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:41.693106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bdd9f-c303-4986-986d-bdfc58e35484 service swift] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:41.693106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bdd9f-c303-4986-986d-bdfc58e35484 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:41.706734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bdd9f-c303-4986-986d-bdfc58e35484 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:41.708480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bdd9f-c303-4986-986d-bdfc58e35484 service swift] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:41.720971 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1303/2593] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:57:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:57:42.549861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb214d2-5314-4f36-b79b-2765bd0814be None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:42.690272 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:42.706346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb214d2-5314-4f36-b79b-2765bd0814be None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:42.711534 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:42.713022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:42.713373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:42.715065 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] PATH_INFO: `/v3/users/5ea28d3129c74588b2c108e339d918ec` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:42.716853 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea28d3129c74588b2c108e339d918ec)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:42.719536 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:42.822288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea28d3129c74588b2c108e339d918ec'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:42.843558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea28d3129c74588b2c108e339d918ec'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:42.864550 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea28d3129c74588b2c108e339d918ec'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:42.878431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea28d3129c74588b2c108e339d918ec'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:42.879236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea28d3129c74588b2c108e339d918ec'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:42.903231 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efb147-7243-468c-baef-c73f3d6423e4 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:42.904153 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efb147-7243-468c-baef-c73f3d6423e4 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:42.904615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efb147-7243-468c-baef-c73f3d6423e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:42.910258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1291/2594] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:57:42 2022] DELETE /identity/v3/users/5ea28d3129c74588b2c108e339d918ec => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:57:42.928381 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-88efb147-7243-468c-baef-c73f3d6423e4 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:42.929808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-88efb147-7243-468c-baef-c73f3d6423e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:42.959456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:43.025869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1304/2595] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:43.043318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:43.046376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:43.165942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e698f-c299-4934-a221-917a3a8a782e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:43.174106 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ace10b-fdab-4723-ae1f-7aea2862c458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:43.175531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ace10b-fdab-4723-ae1f-7aea2862c458 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:43.176263 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ace10b-fdab-4723-ae1f-7aea2862c458 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:43.177051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ace10b-fdab-4723-ae1f-7aea2862c458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:43.196380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ace10b-fdab-4723-ae1f-7aea2862c458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:43.198172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ace10b-fdab-4723-ae1f-7aea2862c458 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:43.210753 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1292/2596] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:57:43.707979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efb147-7243-468c-baef-c73f3d6423e4 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:43.709622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-88efb147-7243-468c-baef-c73f3d6423e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:43.740292 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efb147-7243-468c-baef-c73f3d6423e4 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:43.762518 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:43.763444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:43.763730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:43.764699 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] PATH_INFO: `/v3/projects/b2adde584b004ae6b648453c5477bace` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:43.765884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2adde584b004ae6b648453c5477bace)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:43.768496 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:43.845287 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2adde584b004ae6b648453c5477bace', 'request_id': 'req-83f193a1-84b2-4831-9427-3b1bbdb29ffd'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:43.855064 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2adde584b004ae6b648453c5477bace', 'request_id': 'req-83f193a1-84b2-4831-9427-3b1bbdb29ffd'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:43.863112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2adde584b004ae6b648453c5477bace', 'request_id': 'req-83f193a1-84b2-4831-9427-3b1bbdb29ffd'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:43.879474 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] The token cache is being invalidate because project b2adde584b004ae6b648453c5477bace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:57:43.880355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:43.881075 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:43.882941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1305/2597] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:57:43 2022] DELETE /identity/v3/projects/b2adde584b004ae6b648453c5477bace => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:43.888475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ace10b-fdab-4723-ae1f-7aea2862c458 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:43.902312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ace10b-fdab-4723-ae1f-7aea2862c458 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:43.990087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2221f2-f74b-4850-b0b9-46022ebad614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:43.991265 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2221f2-f74b-4850-b0b9-46022ebad614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:43.991567 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2221f2-f74b-4850-b0b9-46022ebad614 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:43.991846 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2221f2-f74b-4850-b0b9-46022ebad614 admin admin] PATH_INFO: `/v3/users/160c770d29e743c392546b9ea0397da4` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:43.993469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2221f2-f74b-4850-b0b9-46022ebad614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=160c770d29e743c392546b9ea0397da4)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:43.995951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2221f2-f74b-4850-b0b9-46022ebad614 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:44.029952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:44.067914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7a2221f2-f74b-4850-b0b9-46022ebad614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '160c770d29e743c392546b9ea0397da4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:44.082610 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7a2221f2-f74b-4850-b0b9-46022ebad614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '160c770d29e743c392546b9ea0397da4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:44.099615 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7a2221f2-f74b-4850-b0b9-46022ebad614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '160c770d29e743c392546b9ea0397da4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:44.110012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7a2221f2-f74b-4850-b0b9-46022ebad614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '160c770d29e743c392546b9ea0397da4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:44.111309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7a2221f2-f74b-4850-b0b9-46022ebad614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '160c770d29e743c392546b9ea0397da4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:44.120001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1293/2598] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:57:43 2022] DELETE /identity/v3/users/160c770d29e743c392546b9ea0397da4 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:57:44.639629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:44.645155 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:44.670076 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f193a1-84b2-4831-9427-3b1bbdb29ffd admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:44.768840 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:44.769884 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:44.770168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:44.770448 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960 admin admin] PATH_INFO: `/v3/projects/14b5c60f9c7945bea96c2fa3bee5441a` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:44.772115 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14b5c60f9c7945bea96c2fa3bee5441a)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:44.774148 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:44.880890 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14b5c60f9c7945bea96c2fa3bee5441a', 'request_id': 'req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:44.885731 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:44.894378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14b5c60f9c7945bea96c2fa3bee5441a', 'request_id': 'req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:44.904810 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14b5c60f9c7945bea96c2fa3bee5441a', 'request_id': 'req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:44.920700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960 admin admin] The token cache is being invalidate because project 14b5c60f9c7945bea96c2fa3bee5441a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:57:44.921300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:44.922701 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f964c7c5-cc9d-45ef-8d9e-29234c9b7960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:44.924447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1306/2599] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:57:44 2022] DELETE /identity/v3/projects/14b5c60f9c7945bea96c2fa3bee5441a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:57:44.929992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86858c84-2b62-4930-9ac3-c92c256436c0 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:44.930730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86858c84-2b62-4930-9ac3-c92c256436c0 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:44.930730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86858c84-2b62-4930-9ac3-c92c256436c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:44.957925 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-86858c84-2b62-4930-9ac3-c92c256436c0 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:44.959351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-86858c84-2b62-4930-9ac3-c92c256436c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:45.025117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1294/2600] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:57:45.034611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee539cf-0002-45f1-82d1-008c400a54c6 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:45.034889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee539cf-0002-45f1-82d1-008c400a54c6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:45.035168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee539cf-0002-45f1-82d1-008c400a54c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:45.056639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-fee539cf-0002-45f1-82d1-008c400a54c6 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:45.057893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fee539cf-0002-45f1-82d1-008c400a54c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:45.071291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1307/2601] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:45.076261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86858c84-2b62-4930-9ac3-c92c256436c0 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:45.088123 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86858c84-2b62-4930-9ac3-c92c256436c0 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:45.093495 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10ef99-8125-499a-b311-91603c560180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:45.094474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10ef99-8125-499a-b311-91603c560180 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:45.094872 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10ef99-8125-499a-b311-91603c560180 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:45.095249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10ef99-8125-499a-b311-91603c560180 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:45.096098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10ef99-8125-499a-b311-91603c560180 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:57:45.096509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10ef99-8125-499a-b311-91603c560180 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:45.098260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10ef99-8125-499a-b311-91603c560180 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:45.113257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1295/2602] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:57:45.120281 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee539cf-0002-45f1-82d1-008c400a54c6 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:45.134793 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee539cf-0002-45f1-82d1-008c400a54c6 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:45.140135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76270c1-50ac-4732-bf4a-3606a88b19d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:45.142676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76270c1-50ac-4732-bf4a-3606a88b19d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:45.142676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76270c1-50ac-4732-bf4a-3606a88b19d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:45.142676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76270c1-50ac-4732-bf4a-3606a88b19d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:45.142676 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76270c1-50ac-4732-bf4a-3606a88b19d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:45.144007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76270c1-50ac-4732-bf4a-3606a88b19d1 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:45.180048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1308/2603] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:57:45 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:57:45.186714 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10ef99-8125-499a-b311-91603c560180 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:45.199388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10ef99-8125-499a-b311-91603c560180 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:45.270876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecb85d-bbee-49aa-8084-fb2ab7e03235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:45.271625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecb85d-bbee-49aa-8084-fb2ab7e03235 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:45.271973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecb85d-bbee-49aa-8084-fb2ab7e03235 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:45.272256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecb85d-bbee-49aa-8084-fb2ab7e03235 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:45.273029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecb85d-bbee-49aa-8084-fb2ab7e03235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:45.275176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecb85d-bbee-49aa-8084-fb2ab7e03235 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:45.315922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1296/2604] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:57:45 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:57:45.318196 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76270c1-50ac-4732-bf4a-3606a88b19d1 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:45.341116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76270c1-50ac-4732-bf4a-3606a88b19d1 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:45.346327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb80af-5cbc-4d78-8a4e-0af7c54bb9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:45.347282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb80af-5cbc-4d78-8a4e-0af7c54bb9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:45.347575 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb80af-5cbc-4d78-8a4e-0af7c54bb9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:45.348047 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb80af-5cbc-4d78-8a4e-0af7c54bb9d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:45.348808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb80af-5cbc-4d78-8a4e-0af7c54bb9d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:45.350527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb80af-5cbc-4d78-8a4e-0af7c54bb9d1 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:45.363165 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1309/2605] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:57:45.370261 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecb85d-bbee-49aa-8084-fb2ab7e03235 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:45.390780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecb85d-bbee-49aa-8084-fb2ab7e03235 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:45.395622 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2f8df-73f8-4776-bc14-bae2322a0002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:45.396312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2f8df-73f8-4776-bc14-bae2322a0002 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:45.396623 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2f8df-73f8-4776-bc14-bae2322a0002 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:45.396909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2f8df-73f8-4776-bc14-bae2322a0002 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:45.397759 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2f8df-73f8-4776-bc14-bae2322a0002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:45.399945 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2f8df-73f8-4776-bc14-bae2322a0002 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:45.418904 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1297/2606] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:57:45.420140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb80af-5cbc-4d78-8a4e-0af7c54bb9d1 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:45.431965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb80af-5cbc-4d78-8a4e-0af7c54bb9d1 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:45.441624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1e125-9c76-46e4-8d2d-3f82bc856278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:45.442788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1e125-9c76-46e4-8d2d-3f82bc856278 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:45.443159 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1e125-9c76-46e4-8d2d-3f82bc856278 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:45.443514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1e125-9c76-46e4-8d2d-3f82bc856278 admin admin] PATH_INFO: `/v3/projects/2b6abf374b8e435d949b2b9deaf5bfdf/users/e19a7602607e4511b561751382c21c24/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:45.464338 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1e125-9c76-46e4-8d2d-3f82bc856278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b6abf374b8e435d949b2b9deaf5bfdf, user_id=e19a7602607e4511b561751382c21c24, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:45.470042 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1e125-9c76-46e4-8d2d-3f82bc856278 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:45.497796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1310/2607] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:57:45 2022] PUT /identity/v3/projects/2b6abf374b8e435d949b2b9deaf5bfdf/users/e19a7602607e4511b561751382c21c24/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:45.509603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c32d4-ee1f-4787-9600-40f48ea3cdd5 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:45.509944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c32d4-ee1f-4787-9600-40f48ea3cdd5 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:45.510231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c32d4-ee1f-4787-9600-40f48ea3cdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:45.518866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:45.560212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ee0c32d4-ee1f-4787-9600-40f48ea3cdd5 None None] MFA Rules not processed for user `e19a7602607e4511b561751382c21c24`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:45.561774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ee0c32d4-ee1f-4787-9600-40f48ea3cdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:45.666968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1298/2608] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:45.668399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1e125-9c76-46e4-8d2d-3f82bc856278 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:45.765813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1e125-9c76-46e4-8d2d-3f82bc856278 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:45.772322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6db63e-d109-4165-ac9b-7a4bbc78a9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:45.773905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6db63e-d109-4165-ac9b-7a4bbc78a9ac admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:45.774197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6db63e-d109-4165-ac9b-7a4bbc78a9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:45.774480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6db63e-d109-4165-ac9b-7a4bbc78a9ac admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:45.775202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6db63e-d109-4165-ac9b-7a4bbc78a9ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:45.777308 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6db63e-d109-4165-ac9b-7a4bbc78a9ac admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:45.819288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1311/2609] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:57:45 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:57:45.820400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c32d4-ee1f-4787-9600-40f48ea3cdd5 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:45.852892 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c32d4-ee1f-4787-9600-40f48ea3cdd5 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:45.975771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f359f6-bd34-4b44-96d8-97082c3aa1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:45.977596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f359f6-bd34-4b44-96d8-97082c3aa1fa admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:45.977596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f359f6-bd34-4b44-96d8-97082c3aa1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:45.977596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f359f6-bd34-4b44-96d8-97082c3aa1fa admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:45.977596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f359f6-bd34-4b44-96d8-97082c3aa1fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:45.979275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f359f6-bd34-4b44-96d8-97082c3aa1fa admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:46.005302 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1299/2610] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:57:45 2022] POST /identity/v3/users => generated 296 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:57:46.009443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6db63e-d109-4165-ac9b-7a4bbc78a9ac admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:46.021167 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6db63e-d109-4165-ac9b-7a4bbc78a9ac admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:46.030022 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47b6ce-929f-436f-bc99-6ed78e6340a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:46.030717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47b6ce-929f-436f-bc99-6ed78e6340a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:46.031030 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47b6ce-929f-436f-bc99-6ed78e6340a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:46.031834 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47b6ce-929f-436f-bc99-6ed78e6340a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:46.032598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47b6ce-929f-436f-bc99-6ed78e6340a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:46.034252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47b6ce-929f-436f-bc99-6ed78e6340a9 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:46.047214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1312/2611] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:57:46.053227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f359f6-bd34-4b44-96d8-97082c3aa1fa admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:46.064361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f359f6-bd34-4b44-96d8-97082c3aa1fa admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:46.069566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515282d-dfa4-42b4-973e-be8f171f9b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:46.070822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515282d-dfa4-42b4-973e-be8f171f9b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:46.071199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515282d-dfa4-42b4-973e-be8f171f9b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:46.071590 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515282d-dfa4-42b4-973e-be8f171f9b69 admin admin] PATH_INFO: `/v3/projects/e441e3cd7a26443eb163de557cd51b29/users/d1f3cdc73f154b02bd4c0199c82bde57/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:46.099085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515282d-dfa4-42b4-973e-be8f171f9b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e441e3cd7a26443eb163de557cd51b29, user_id=d1f3cdc73f154b02bd4c0199c82bde57, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:46.101856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515282d-dfa4-42b4-973e-be8f171f9b69 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:46.120996 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1300/2612] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:57:46 2022] PUT /identity/v3/projects/e441e3cd7a26443eb163de557cd51b29/users/d1f3cdc73f154b02bd4c0199c82bde57/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:46.136649 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47b6ce-929f-436f-bc99-6ed78e6340a9 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:46.154375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47b6ce-929f-436f-bc99-6ed78e6340a9 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:46.225922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:46.241158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fad25-005e-4121-a6a4-7f54fa116846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:46.241976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fad25-005e-4121-a6a4-7f54fa116846 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:46.242821 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fad25-005e-4121-a6a4-7f54fa116846 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:46.243186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fad25-005e-4121-a6a4-7f54fa116846 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:46.244153 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fad25-005e-4121-a6a4-7f54fa116846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:46.246698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fad25-005e-4121-a6a4-7f54fa116846 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:46.262160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1313/2613] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:57:46.268275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515282d-dfa4-42b4-973e-be8f171f9b69 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:46.280659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515282d-dfa4-42b4-973e-be8f171f9b69 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:46.286554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997b84c-6d8e-492d-8047-88683b35d3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:46.287480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997b84c-6d8e-492d-8047-88683b35d3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:46.287811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997b84c-6d8e-492d-8047-88683b35d3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:46.288648 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997b84c-6d8e-492d-8047-88683b35d3bf admin admin] PATH_INFO: `/v3/projects/e441e3cd7a26443eb163de557cd51b29/users/d1f3cdc73f154b02bd4c0199c82bde57/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:46.290835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997b84c-6d8e-492d-8047-88683b35d3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e441e3cd7a26443eb163de557cd51b29, user_id=d1f3cdc73f154b02bd4c0199c82bde57, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:46.295810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997b84c-6d8e-492d-8047-88683b35d3bf admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:46.312912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1301/2614] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:57:46 2022] PUT /identity/v3/projects/e441e3cd7a26443eb163de557cd51b29/users/d1f3cdc73f154b02bd4c0199c82bde57/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:46.322663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e380c-9a7c-4cef-8fa3-ad5dccd1f06f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:46.322979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e380c-9a7c-4cef-8fa3-ad5dccd1f06f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:46.323262 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e380c-9a7c-4cef-8fa3-ad5dccd1f06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:46.368658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-394e380c-9a7c-4cef-8fa3-ad5dccd1f06f None None] MFA Rules not processed for user `d1f3cdc73f154b02bd4c0199c82bde57`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:46.370679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-394e380c-9a7c-4cef-8fa3-ad5dccd1f06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:46.455794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1314/2615] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:46.470255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997b84c-6d8e-492d-8047-88683b35d3bf admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:46.475528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d997b84c-6d8e-492d-8047-88683b35d3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:46.534279 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:46.551835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997b84c-6d8e-492d-8047-88683b35d3bf admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:46.555679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e8e19-9a90-4f3e-a532-2637a1e9e2c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:46.556350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e8e19-9a90-4f3e-a532-2637a1e9e2c7 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:46.556666 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e8e19-9a90-4f3e-a532-2637a1e9e2c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:46.556944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e8e19-9a90-4f3e-a532-2637a1e9e2c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:46.570769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e8e19-9a90-4f3e-a532-2637a1e9e2c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:46.572505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e8e19-9a90-4f3e-a532-2637a1e9e2c7 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:46.588677 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1302/2616] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:57:47.153504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e380c-9a7c-4cef-8fa3-ad5dccd1f06f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:47.227407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e380c-9a7c-4cef-8fa3-ad5dccd1f06f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:47.242995 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:47.243968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:47.244229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:47.244960 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] PATH_INFO: `/v3/users/e19a7602607e4511b561751382c21c24` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:47.246447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e19a7602607e4511b561751382c21c24)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:47.248269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:47.313116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e19a7602607e4511b561751382c21c24'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:47.313903 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e19a7602607e4511b561751382c21c24'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:47.330464 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e19a7602607e4511b561751382c21c24'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:47.348167 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e19a7602607e4511b561751382c21c24'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:47.351120 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:47.358152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e19a7602607e4511b561751382c21c24'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:47.373760 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1315/2617] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:57:47 2022] DELETE /identity/v3/users/e19a7602607e4511b561751382c21c24 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:47.378881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103849c-7593-4f28-a052-4d7c217e88de None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:47.379508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103849c-7593-4f28-a052-4d7c217e88de None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:47.379508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103849c-7593-4f28-a052-4d7c217e88de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:47.401580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-e103849c-7593-4f28-a052-4d7c217e88de None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:47.403082 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e103849c-7593-4f28-a052-4d7c217e88de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:47.484056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1303/2618] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:47.496694 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:47.498666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:47.590606 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492e629-74c7-4b71-97dc-a4d25b854480 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:47.591339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146873d2-52e3-437d-b559-fcc31b5f5810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:47.592119 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146873d2-52e3-437d-b559-fcc31b5f5810 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:47.592494 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146873d2-52e3-437d-b559-fcc31b5f5810 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:47.594994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146873d2-52e3-437d-b559-fcc31b5f5810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:47.607265 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146873d2-52e3-437d-b559-fcc31b5f5810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:47.610135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146873d2-52e3-437d-b559-fcc31b5f5810 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:47.622035 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1316/2619] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:57:47.954932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103849c-7593-4f28-a052-4d7c217e88de None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:47.956856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e103849c-7593-4f28-a052-4d7c217e88de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:47.972376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103849c-7593-4f28-a052-4d7c217e88de None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:47.977399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:47.978369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:47.978647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:47.978918 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] PATH_INFO: `/v3/projects/2b6abf374b8e435d949b2b9deaf5bfdf` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:47.980260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b6abf374b8e435d949b2b9deaf5bfdf)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:47.982369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:48.040999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b6abf374b8e435d949b2b9deaf5bfdf', 'request_id': 'req-a10deb6a-f4a5-429d-9370-1eee4ff18f96'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.049887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b6abf374b8e435d949b2b9deaf5bfdf', 'request_id': 'req-a10deb6a-f4a5-429d-9370-1eee4ff18f96'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.056482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b6abf374b8e435d949b2b9deaf5bfdf', 'request_id': 'req-a10deb6a-f4a5-429d-9370-1eee4ff18f96'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.072286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] The token cache is being invalidate because project 2b6abf374b8e435d949b2b9deaf5bfdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:57:48.072608 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.073644 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.075738 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1304/2620] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:57:47 2022] DELETE /identity/v3/projects/2b6abf374b8e435d949b2b9deaf5bfdf => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:48.080388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146873d2-52e3-437d-b559-fcc31b5f5810 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:48.090888 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146873d2-52e3-437d-b559-fcc31b5f5810 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:48.180915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9f5dc-1a21-4dc8-b417-cd8bbab90693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:48.182463 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9f5dc-1a21-4dc8-b417-cd8bbab90693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:48.182913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9f5dc-1a21-4dc8-b417-cd8bbab90693 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:48.183273 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9f5dc-1a21-4dc8-b417-cd8bbab90693 admin admin] PATH_INFO: `/v3/users/d1f3cdc73f154b02bd4c0199c82bde57` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:48.184848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b9f5dc-1a21-4dc8-b417-cd8bbab90693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1f3cdc73f154b02bd4c0199c82bde57)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:48.187127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b9f5dc-1a21-4dc8-b417-cd8bbab90693 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:48.253995 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f9b9f5dc-1a21-4dc8-b417-cd8bbab90693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1f3cdc73f154b02bd4c0199c82bde57'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.254791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f9b9f5dc-1a21-4dc8-b417-cd8bbab90693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1f3cdc73f154b02bd4c0199c82bde57'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.263864 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f9b9f5dc-1a21-4dc8-b417-cd8bbab90693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1f3cdc73f154b02bd4c0199c82bde57'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.280718 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f9b9f5dc-1a21-4dc8-b417-cd8bbab90693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1f3cdc73f154b02bd4c0199c82bde57'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.295710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f9b9f5dc-1a21-4dc8-b417-cd8bbab90693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1f3cdc73f154b02bd4c0199c82bde57'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.315491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1317/2621] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:57:48 2022] DELETE /identity/v3/users/d1f3cdc73f154b02bd4c0199c82bde57 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:48.701537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:48.703557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:48.720694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10deb6a-f4a5-429d-9370-1eee4ff18f96 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:48.799491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eafd89-7f19-4ca7-83ac-3c49d2171410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:48.800501 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eafd89-7f19-4ca7-83ac-3c49d2171410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:48.800815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eafd89-7f19-4ca7-83ac-3c49d2171410 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:48.801095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eafd89-7f19-4ca7-83ac-3c49d2171410 admin admin] PATH_INFO: `/v3/projects/e441e3cd7a26443eb163de557cd51b29` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:48.802487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eafd89-7f19-4ca7-83ac-3c49d2171410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e441e3cd7a26443eb163de557cd51b29)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:48.805101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eafd89-7f19-4ca7-83ac-3c49d2171410 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:48.829970 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:48.883304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-49eafd89-7f19-4ca7-83ac-3c49d2171410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e441e3cd7a26443eb163de557cd51b29', 'request_id': 'req-49eafd89-7f19-4ca7-83ac-3c49d2171410'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.895377 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-49eafd89-7f19-4ca7-83ac-3c49d2171410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e441e3cd7a26443eb163de557cd51b29', 'request_id': 'req-49eafd89-7f19-4ca7-83ac-3c49d2171410'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.900936 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-49eafd89-7f19-4ca7-83ac-3c49d2171410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e441e3cd7a26443eb163de557cd51b29', 'request_id': 'req-49eafd89-7f19-4ca7-83ac-3c49d2171410'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.916439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-49eafd89-7f19-4ca7-83ac-3c49d2171410 admin admin] The token cache is being invalidate because project e441e3cd7a26443eb163de557cd51b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:57:48.916793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-49eafd89-7f19-4ca7-83ac-3c49d2171410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.917828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-49eafd89-7f19-4ca7-83ac-3c49d2171410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:57:48.920684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1305/2622] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:57:48 2022] DELETE /identity/v3/projects/e441e3cd7a26443eb163de557cd51b29 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:48.926566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c1297-240d-4ce3-9bb2-f05d325d07e5 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:48.926842 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c1297-240d-4ce3-9bb2-f05d325d07e5 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:48.927234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c1297-240d-4ce3-9bb2-f05d325d07e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:48.950710 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-f74c1297-240d-4ce3-9bb2-f05d325d07e5 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:48.952052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f74c1297-240d-4ce3-9bb2-f05d325d07e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:48.963307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:49.051089 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1318/2623] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:49.064399 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e745d0c-1821-46d6-b65d-ef4bb0d7dc9f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:49.064985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e745d0c-1821-46d6-b65d-ef4bb0d7dc9f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:49.065452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e745d0c-1821-46d6-b65d-ef4bb0d7dc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:49.089496 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-0e745d0c-1821-46d6-b65d-ef4bb0d7dc9f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:49.090765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0e745d0c-1821-46d6-b65d-ef4bb0d7dc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:49.099206 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1306/2624] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:49.103650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c1297-240d-4ce3-9bb2-f05d325d07e5 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:49.115595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c1297-240d-4ce3-9bb2-f05d325d07e5 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:49.120785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889220de-eeb7-4239-bd71-88327c1721fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:49.121647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889220de-eeb7-4239-bd71-88327c1721fe admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:49.121777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889220de-eeb7-4239-bd71-88327c1721fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:49.122012 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889220de-eeb7-4239-bd71-88327c1721fe admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:49.122752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889220de-eeb7-4239-bd71-88327c1721fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:57:49.123090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889220de-eeb7-4239-bd71-88327c1721fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:49.124741 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889220de-eeb7-4239-bd71-88327c1721fe admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:49.140515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1319/2625] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:57:49.144927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e745d0c-1821-46d6-b65d-ef4bb0d7dc9f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:49.155015 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e745d0c-1821-46d6-b65d-ef4bb0d7dc9f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:49.159977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d6d6a-fd86-4bbf-a1dc-079888a113ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:49.160684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d6d6a-fd86-4bbf-a1dc-079888a113ec admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:49.160972 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d6d6a-fd86-4bbf-a1dc-079888a113ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:49.161246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d6d6a-fd86-4bbf-a1dc-079888a113ec admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:49.161949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d6d6a-fd86-4bbf-a1dc-079888a113ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:49.163878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d6d6a-fd86-4bbf-a1dc-079888a113ec admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:49.226013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1307/2626] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:57:49 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:57:49.230115 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889220de-eeb7-4239-bd71-88327c1721fe admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:49.241796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889220de-eeb7-4239-bd71-88327c1721fe admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:49.320523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d653c0-35e7-4c17-9440-c77663ce987a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:49.321398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d653c0-35e7-4c17-9440-c77663ce987a admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:49.321782 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d653c0-35e7-4c17-9440-c77663ce987a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:49.322154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d653c0-35e7-4c17-9440-c77663ce987a admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:49.322986 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d653c0-35e7-4c17-9440-c77663ce987a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:49.326101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d653c0-35e7-4c17-9440-c77663ce987a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:49.352609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1320/2627] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:57:49 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:57:49.355262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d6d6a-fd86-4bbf-a1dc-079888a113ec admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:49.368899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d6d6a-fd86-4bbf-a1dc-079888a113ec admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:49.390530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac4467-5f54-4a9e-9645-edca5a1b2569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:49.391237 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac4467-5f54-4a9e-9645-edca5a1b2569 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:49.391525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac4467-5f54-4a9e-9645-edca5a1b2569 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:49.391867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac4467-5f54-4a9e-9645-edca5a1b2569 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:49.392595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac4467-5f54-4a9e-9645-edca5a1b2569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:49.400486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac4467-5f54-4a9e-9645-edca5a1b2569 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:49.424364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1308/2628] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:57:49.430306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d653c0-35e7-4c17-9440-c77663ce987a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:49.455110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d653c0-35e7-4c17-9440-c77663ce987a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:49.461565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5060d-076d-4554-ba6b-23d69e42bccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:49.462417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5060d-076d-4554-ba6b-23d69e42bccd admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:49.463204 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5060d-076d-4554-ba6b-23d69e42bccd admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:49.463700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5060d-076d-4554-ba6b-23d69e42bccd admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:49.468314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5060d-076d-4554-ba6b-23d69e42bccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:49.468314 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5060d-076d-4554-ba6b-23d69e42bccd admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:49.481848 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1321/2629] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:57:49.487543 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac4467-5f54-4a9e-9645-edca5a1b2569 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:49.504485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac4467-5f54-4a9e-9645-edca5a1b2569 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:49.514317 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b2822-f93e-4a1d-8fbd-a29b6933343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:49.515966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b2822-f93e-4a1d-8fbd-a29b6933343c admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:49.516613 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b2822-f93e-4a1d-8fbd-a29b6933343c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:49.517204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b2822-f93e-4a1d-8fbd-a29b6933343c admin admin] PATH_INFO: `/v3/projects/cabc58a85dcf4be8a8ad4d21190ca053/users/2a621ef51f734f928ec7c2d25efad577/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:49.566216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b2822-f93e-4a1d-8fbd-a29b6933343c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cabc58a85dcf4be8a8ad4d21190ca053, user_id=2a621ef51f734f928ec7c2d25efad577, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:49.573150 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b2822-f93e-4a1d-8fbd-a29b6933343c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:49.601895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1309/2630] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:57:49 2022] PUT /identity/v3/projects/cabc58a85dcf4be8a8ad4d21190ca053/users/2a621ef51f734f928ec7c2d25efad577/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:57:49.608613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc47ff-e578-461c-99d2-8eb933769bde None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:49.608930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc47ff-e578-461c-99d2-8eb933769bde None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:49.609240 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc47ff-e578-461c-99d2-8eb933769bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:49.630728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-25bc47ff-e578-461c-99d2-8eb933769bde None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:49.632001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-25bc47ff-e578-461c-99d2-8eb933769bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:49.728472 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1322/2631] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:49.740496 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b2822-f93e-4a1d-8fbd-a29b6933343c admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:49.742905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-686b2822-f93e-4a1d-8fbd-a29b6933343c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:49.836457 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b2822-f93e-4a1d-8fbd-a29b6933343c admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:49.841258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c1f37-7b83-4e2c-a0df-8bf40c1c5cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:49.842002 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c1f37-7b83-4e2c-a0df-8bf40c1c5cab service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:49.842295 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c1f37-7b83-4e2c-a0df-8bf40c1c5cab service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:49.843287 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c1f37-7b83-4e2c-a0df-8bf40c1c5cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:49.857456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c1f37-7b83-4e2c-a0df-8bf40c1c5cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:49.858934 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c1f37-7b83-4e2c-a0df-8bf40c1c5cab service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:49.875241 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1310/2632] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:57:50.791520 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0d114-5751-4a50-bc97-6f2bc24ffaf6 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:50.792146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0d114-5751-4a50-bc97-6f2bc24ffaf6 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:50.792374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0d114-5751-4a50-bc97-6f2bc24ffaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:50.823860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-1bc0d114-5751-4a50-bc97-6f2bc24ffaf6 None None] MFA Rules not processed for user `2a621ef51f734f928ec7c2d25efad577`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:50.825379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-1bc0d114-5751-4a50-bc97-6f2bc24ffaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:50.835989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:50.926932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1323/2633] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:50.931961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c1f37-7b83-4e2c-a0df-8bf40c1c5cab service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:50.944989 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c1f37-7b83-4e2c-a0df-8bf40c1c5cab service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:50.949938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71dc4f-990e-4320-9abf-2ae8c17ec789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:50.950602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71dc4f-990e-4320-9abf-2ae8c17ec789 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:50.950882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71dc4f-990e-4320-9abf-2ae8c17ec789 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:50.951153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71dc4f-990e-4320-9abf-2ae8c17ec789 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:50.951884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71dc4f-990e-4320-9abf-2ae8c17ec789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:50.954529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71dc4f-990e-4320-9abf-2ae8c17ec789 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:50.985855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1311/2634] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:57:50 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:57:50.987161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0d114-5751-4a50-bc97-6f2bc24ffaf6 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:50.998750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0d114-5751-4a50-bc97-6f2bc24ffaf6 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:51.077186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257136b-3e98-4e89-bed8-cc37a34e8669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:51.077918 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257136b-3e98-4e89-bed8-cc37a34e8669 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:51.078718 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257136b-3e98-4e89-bed8-cc37a34e8669 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:51.078994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257136b-3e98-4e89-bed8-cc37a34e8669 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:51.079730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257136b-3e98-4e89-bed8-cc37a34e8669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:51.081739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257136b-3e98-4e89-bed8-cc37a34e8669 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:51.107282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1324/2635] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:57:50 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:57:51.108436 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71dc4f-990e-4320-9abf-2ae8c17ec789 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:51.119891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71dc4f-990e-4320-9abf-2ae8c17ec789 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:51.124519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71536c-70ee-481d-9e80-9fc2717c4403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:51.125774 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71536c-70ee-481d-9e80-9fc2717c4403 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:51.126058 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71536c-70ee-481d-9e80-9fc2717c4403 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:51.126333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71536c-70ee-481d-9e80-9fc2717c4403 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:51.127061 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa71536c-70ee-481d-9e80-9fc2717c4403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:51.128685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa71536c-70ee-481d-9e80-9fc2717c4403 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:51.140488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1312/2636] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:57:51.144777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257136b-3e98-4e89-bed8-cc37a34e8669 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:51.155583 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257136b-3e98-4e89-bed8-cc37a34e8669 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:51.160496 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d96be2-0e84-4ad0-b07a-bf114766fdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:51.161419 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d96be2-0e84-4ad0-b07a-bf114766fdd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:51.161708 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d96be2-0e84-4ad0-b07a-bf114766fdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:51.161988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d96be2-0e84-4ad0-b07a-bf114766fdd8 admin admin] PATH_INFO: `/v3/projects/bbf0e56496a547f7840d5463522df29c/users/649ca5c034a14e569f7d671823de3803/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:51.178697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d96be2-0e84-4ad0-b07a-bf114766fdd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf0e56496a547f7840d5463522df29c, user_id=649ca5c034a14e569f7d671823de3803, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:51.181639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d96be2-0e84-4ad0-b07a-bf114766fdd8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:51.197675 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1325/2637] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:57:51 2022] PUT /identity/v3/projects/bbf0e56496a547f7840d5463522df29c/users/649ca5c034a14e569f7d671823de3803/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:57:51.201804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71536c-70ee-481d-9e80-9fc2717c4403 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:51.212661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71536c-70ee-481d-9e80-9fc2717c4403 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:51.216366 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:51.279618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abc957-8efc-4545-ac2f-664760af2c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:51.280301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abc957-8efc-4545-ac2f-664760af2c2e admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:51.280626 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abc957-8efc-4545-ac2f-664760af2c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:51.280912 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abc957-8efc-4545-ac2f-664760af2c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:51.282313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abc957-8efc-4545-ac2f-664760af2c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:51.283943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abc957-8efc-4545-ac2f-664760af2c2e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:51.296333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1313/2638] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:57:51.301482 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d96be2-0e84-4ad0-b07a-bf114766fdd8 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:51.313245 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d96be2-0e84-4ad0-b07a-bf114766fdd8 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:51.317324 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1556cf-ab76-4b50-ad04-96b1a06a7de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:51.318222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1556cf-ab76-4b50-ad04-96b1a06a7de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:51.318508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1556cf-ab76-4b50-ad04-96b1a06a7de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:51.318793 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1556cf-ab76-4b50-ad04-96b1a06a7de7 admin admin] PATH_INFO: `/v3/projects/bbf0e56496a547f7840d5463522df29c/users/649ca5c034a14e569f7d671823de3803/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:51.320958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1556cf-ab76-4b50-ad04-96b1a06a7de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf0e56496a547f7840d5463522df29c, user_id=649ca5c034a14e569f7d671823de3803, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:51.323599 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1556cf-ab76-4b50-ad04-96b1a06a7de7 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:51.335114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1326/2639] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:57:51 2022] PUT /identity/v3/projects/bbf0e56496a547f7840d5463522df29c/users/649ca5c034a14e569f7d671823de3803/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:57:52.040834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba0d4b-61ff-4f94-9975-5c34592f7d98 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:52.041422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba0d4b-61ff-4f94-9975-5c34592f7d98 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:52.041422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba0d4b-61ff-4f94-9975-5c34592f7d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:52.076839 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-92ba0d4b-61ff-4f94-9975-5c34592f7d98 None None] MFA Rules not processed for user `649ca5c034a14e569f7d671823de3803`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:57:52.078633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-92ba0d4b-61ff-4f94-9975-5c34592f7d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:52.173441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1314/2640] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:57:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:57:52.185984 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1556cf-ab76-4b50-ad04-96b1a06a7de7 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:52.188788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ed1556cf-ab76-4b50-ad04-96b1a06a7de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:57:52.259376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1556cf-ab76-4b50-ad04-96b1a06a7de7 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:52.263163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdb396-af7d-40b2-b439-a6471ef3ac1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:52.263833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdb396-af7d-40b2-b439-a6471ef3ac1b service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:52.264112 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdb396-af7d-40b2-b439-a6471ef3ac1b service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:52.264391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdb396-af7d-40b2-b439-a6471ef3ac1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:52.278405 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:57:52.336807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffdb396-af7d-40b2-b439-a6471ef3ac1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:52.339406 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffdb396-af7d-40b2-b439-a6471ef3ac1b service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:52.353322 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1327/2641] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:57:54.816051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba0d4b-61ff-4f94-9975-5c34592f7d98 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:57:54.826007 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba0d4b-61ff-4f94-9975-5c34592f7d98 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:57:54.830180 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7049bf-9cd9-4abd-8d83-31c352a82c9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:57:54.830861 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7049bf-9cd9-4abd-8d83-31c352a82c9a service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:57:54.831140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7049bf-9cd9-4abd-8d83-31c352a82c9a service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:57:54.831415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7049bf-9cd9-4abd-8d83-31c352a82c9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:57:54.841640 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7049bf-9cd9-4abd-8d83-31c352a82c9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:57:54.843096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7049bf-9cd9-4abd-8d83-31c352a82c9a service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:57:54.853844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1315/2642] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:57:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:58:02.346962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=20 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:03.244118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4465655-5128-4cd0-96ad-408d6b4a6e55 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:03.244454 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4465655-5128-4cd0-96ad-408d6b4a6e55 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:03.245018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4465655-5128-4cd0-96ad-408d6b4a6e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:03.267858 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-b4465655-5128-4cd0-96ad-408d6b4a6e55 None None] MFA Rules not processed for user `c151c53202f24faeb3accf802716652f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:03.269626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b4465655-5128-4cd0-96ad-408d6b4a6e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:03.331375 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1328/2643] 10.222.0.52 () {58 vars in 967 bytes} [Mon Jul 25 08:58:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:03.343937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7049bf-9cd9-4abd-8d83-31c352a82c9a service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:03.345451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1d7049bf-9cd9-4abd-8d83-31c352a82c9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:03.414439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7049bf-9cd9-4abd-8d83-31c352a82c9a service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:03.419479 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd571d3f-f4a5-419b-bfb9-b074064b0cc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d6be43abd85417c9fb7bbcb36a1f77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:03.420133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd571d3f-f4a5-419b-bfb9-b074064b0cc2 service swift] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:03.420415 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd571d3f-f4a5-419b-bfb9-b074064b0cc2 service swift] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:03.420752 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd571d3f-f4a5-419b-bfb9-b074064b0cc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:03.429438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd571d3f-f4a5-419b-bfb9-b074064b0cc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:03.431050 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd571d3f-f4a5-419b-bfb9-b074064b0cc2 service swift] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:03.440537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1316/2644] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Jul 25 08:58:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 08:58:08.109988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4465655-5128-4cd0-96ad-408d6b4a6e55 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:08.111497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b4465655-5128-4cd0-96ad-408d6b4a6e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:08.193288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4465655-5128-4cd0-96ad-408d6b4a6e55 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:08.202437 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542a43b-c656-4721-9a4c-f123fa58287e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:08.203181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542a43b-c656-4721-9a4c-f123fa58287e service placement] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:08.203475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542a43b-c656-4721-9a4c-f123fa58287e service placement] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:08.203785 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542a43b-c656-4721-9a4c-f123fa58287e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:08.205861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a542a43b-c656-4721-9a4c-f123fa58287e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:08.219826 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542a43b-c656-4721-9a4c-f123fa58287e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:08.221613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542a43b-c656-4721-9a4c-f123fa58287e service placement] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:08.232546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1329/2645] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:58:08.709895 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd571d3f-f4a5-419b-bfb9-b074064b0cc2 service swift] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:08.715558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:08.777520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd571d3f-f4a5-419b-bfb9-b074064b0cc2 service swift] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:08.782093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:08.782959 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:08.783236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:08.783505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] PATH_INFO: `/v3/users/2a621ef51f734f928ec7c2d25efad577` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:08.784750 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a621ef51f734f928ec7c2d25efad577)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:08.786653 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:08.832338 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a621ef51f734f928ec7c2d25efad577'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:08.840182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a621ef51f734f928ec7c2d25efad577'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:08.852098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a621ef51f734f928ec7c2d25efad577'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:08.858909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a621ef51f734f928ec7c2d25efad577'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:08.859591 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a621ef51f734f928ec7c2d25efad577'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:08.865489 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1317/2646] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:58:08 2022] DELETE /identity/v3/users/2a621ef51f734f928ec7c2d25efad577 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:09.045580 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542a43b-c656-4721-9a4c-f123fa58287e service placement] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:09.058070 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542a43b-c656-4721-9a4c-f123fa58287e service placement] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:09.127783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:09.127783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:09.136369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:09.136369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] PATH_INFO: `/v3/projects/cabc58a85dcf4be8a8ad4d21190ca053` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:09.136369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cabc58a85dcf4be8a8ad4d21190ca053)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:09.136369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:09.175853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:09.187690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cabc58a85dcf4be8a8ad4d21190ca053', 'request_id': 'req-b83b3e02-73e4-434f-806c-a8f99bffb1c4'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.194877 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cabc58a85dcf4be8a8ad4d21190ca053', 'request_id': 'req-b83b3e02-73e4-434f-806c-a8f99bffb1c4'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.200849 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cabc58a85dcf4be8a8ad4d21190ca053', 'request_id': 'req-b83b3e02-73e4-434f-806c-a8f99bffb1c4'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.213549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] The token cache is being invalidate because project cabc58a85dcf4be8a8ad4d21190ca053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:58:09.213938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.214650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.216243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1330/2647] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:58:09 2022] DELETE /identity/v3/projects/cabc58a85dcf4be8a8ad4d21190ca053 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:09.220842 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:09.222333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:09.235423 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae18b2-47fd-42cb-8251-b7c225a87cec admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:09.291967 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa284b58-4c32-46ff-8aaa-b361f0df5134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:09.293055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa284b58-4c32-46ff-8aaa-b361f0df5134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:09.293403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa284b58-4c32-46ff-8aaa-b361f0df5134 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:09.293747 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa284b58-4c32-46ff-8aaa-b361f0df5134 admin admin] PATH_INFO: `/v3/users/649ca5c034a14e569f7d671823de3803` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:09.295103 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa284b58-4c32-46ff-8aaa-b361f0df5134 admin admin] RBAC: Authorizing `identity:delete_user(user_id=649ca5c034a14e569f7d671823de3803)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:09.297146 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa284b58-4c32-46ff-8aaa-b361f0df5134 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:09.333074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:09.356145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-fa284b58-4c32-46ff-8aaa-b361f0df5134 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '649ca5c034a14e569f7d671823de3803'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.363676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-fa284b58-4c32-46ff-8aaa-b361f0df5134 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '649ca5c034a14e569f7d671823de3803'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.376827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-fa284b58-4c32-46ff-8aaa-b361f0df5134 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '649ca5c034a14e569f7d671823de3803'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.383624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-fa284b58-4c32-46ff-8aaa-b361f0df5134 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '649ca5c034a14e569f7d671823de3803'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.384265 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-fa284b58-4c32-46ff-8aaa-b361f0df5134 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '649ca5c034a14e569f7d671823de3803'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.390315 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1318/2648] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:58:09 2022] DELETE /identity/v3/users/649ca5c034a14e569f7d671823de3803 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:09.570138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:09.571621 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:09.588407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b3e02-73e4-434f-806c-a8f99bffb1c4 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:09.647355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:09.648227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:09.648512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:09.648905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a admin admin] PATH_INFO: `/v3/projects/bbf0e56496a547f7840d5463522df29c` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:09.650127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbf0e56496a547f7840d5463522df29c)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:09.652049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:09.699414 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbf0e56496a547f7840d5463522df29c', 'request_id': 'req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.707499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbf0e56496a547f7840d5463522df29c', 'request_id': 'req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.722756 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbf0e56496a547f7840d5463522df29c', 'request_id': 'req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.734192 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a admin admin] The token cache is being invalidate because project bbf0e56496a547f7840d5463522df29c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:58:09.734499 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.735160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-2bed3347-1f78-4092-b6a9-4dd5958ecc0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:09.736753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1331/2649] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:58:09 2022] DELETE /identity/v3/projects/bbf0e56496a547f7840d5463522df29c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:09.741705 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73edaed-823e-4c06-81fe-85181a091e92 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:09.742173 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73edaed-823e-4c06-81fe-85181a091e92 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:09.742290 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73edaed-823e-4c06-81fe-85181a091e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:09.762858 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-f73edaed-823e-4c06-81fe-85181a091e92 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:09.764079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f73edaed-823e-4c06-81fe-85181a091e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:09.826037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1319/2650] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:09.833994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f4531-ca15-4324-9602-3ca663413ef9 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:09.834294 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f4531-ca15-4324-9602-3ca663413ef9 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:09.834578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f4531-ca15-4324-9602-3ca663413ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:09.854402 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-563f4531-ca15-4324-9602-3ca663413ef9 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:09.855626 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-563f4531-ca15-4324-9602-3ca663413ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:09.864303 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1332/2651] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:09.868817 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73edaed-823e-4c06-81fe-85181a091e92 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:09.880239 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73edaed-823e-4c06-81fe-85181a091e92 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:09.885070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc44e0d-7927-4a68-9e9d-15ef096b646d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:09.885755 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc44e0d-7927-4a68-9e9d-15ef096b646d admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:09.886033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc44e0d-7927-4a68-9e9d-15ef096b646d admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:09.886308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc44e0d-7927-4a68-9e9d-15ef096b646d admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:09.887052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc44e0d-7927-4a68-9e9d-15ef096b646d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:58:09.887354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc44e0d-7927-4a68-9e9d-15ef096b646d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:09.889049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc44e0d-7927-4a68-9e9d-15ef096b646d admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:09.903517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1320/2652] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:58:09.908087 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f4531-ca15-4324-9602-3ca663413ef9 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:09.918005 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f4531-ca15-4324-9602-3ca663413ef9 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:09.923300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec65e3-c96a-46a6-b00b-d0aa12575164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:09.923981 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec65e3-c96a-46a6-b00b-d0aa12575164 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:09.924271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec65e3-c96a-46a6-b00b-d0aa12575164 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:09.924552 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec65e3-c96a-46a6-b00b-d0aa12575164 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:09.925317 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec65e3-c96a-46a6-b00b-d0aa12575164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:09.927274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec65e3-c96a-46a6-b00b-d0aa12575164 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:09.954811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1333/2653] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:58:09 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:58:09.959177 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc44e0d-7927-4a68-9e9d-15ef096b646d admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:09.969566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc44e0d-7927-4a68-9e9d-15ef096b646d admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:10.028270 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad5213-324e-41c7-a39a-e33e86a83dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:10.029094 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad5213-324e-41c7-a39a-e33e86a83dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.029384 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad5213-324e-41c7-a39a-e33e86a83dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.029670 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad5213-324e-41c7-a39a-e33e86a83dc8 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.030372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad5213-324e-41c7-a39a-e33e86a83dc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:10.032554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad5213-324e-41c7-a39a-e33e86a83dc8 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:10.054908 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1321/2654] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:58:09 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:58:10.059218 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec65e3-c96a-46a6-b00b-d0aa12575164 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:10.070958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec65e3-c96a-46a6-b00b-d0aa12575164 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:10.075704 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeab790-7bb1-4667-ac02-7cb11453dde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:10.076369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeab790-7bb1-4667-ac02-7cb11453dde1 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.076714 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeab790-7bb1-4667-ac02-7cb11453dde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.077000 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeab790-7bb1-4667-ac02-7cb11453dde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.077736 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeab790-7bb1-4667-ac02-7cb11453dde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:10.079366 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeab790-7bb1-4667-ac02-7cb11453dde1 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:10.090022 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1334/2655] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:10.094099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad5213-324e-41c7-a39a-e33e86a83dc8 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:10.104264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad5213-324e-41c7-a39a-e33e86a83dc8 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:10.111443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be151e7f-bd60-4504-9414-a675d0029a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:10.112414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be151e7f-bd60-4504-9414-a675d0029a96 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.112892 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be151e7f-bd60-4504-9414-a675d0029a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.113309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be151e7f-bd60-4504-9414-a675d0029a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.114361 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be151e7f-bd60-4504-9414-a675d0029a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:10.116350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be151e7f-bd60-4504-9414-a675d0029a96 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:10.128198 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1322/2656] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:58:10.132742 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeab790-7bb1-4667-ac02-7cb11453dde1 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:10.143137 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeab790-7bb1-4667-ac02-7cb11453dde1 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:10.147854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afd1ae-efdd-45df-8427-23484174610c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:10.148773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afd1ae-efdd-45df-8427-23484174610c admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.149062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afd1ae-efdd-45df-8427-23484174610c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.149342 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afd1ae-efdd-45df-8427-23484174610c admin admin] PATH_INFO: `/v3/projects/18245c11e5f549d083d1adc4d3fe4d9c/users/2c570307fc5647159324bf3794201976/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.162753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:10.165972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afd1ae-efdd-45df-8427-23484174610c admin admin] RBAC: Authorizing `identity:create_grant(project_id=18245c11e5f549d083d1adc4d3fe4d9c, user_id=2c570307fc5647159324bf3794201976, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:10.168976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afd1ae-efdd-45df-8427-23484174610c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:10.184334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1335/2657] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:58:10 2022] PUT /identity/v3/projects/18245c11e5f549d083d1adc4d3fe4d9c/users/2c570307fc5647159324bf3794201976/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:10.196487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f01a7-06a8-4e50-ac3f-8b44dbda9d36 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.196815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f01a7-06a8-4e50-ac3f-8b44dbda9d36 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.197102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f01a7-06a8-4e50-ac3f-8b44dbda9d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.216691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:10.230472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-ef3f01a7-06a8-4e50-ac3f-8b44dbda9d36 None None] MFA Rules not processed for user `2c570307fc5647159324bf3794201976`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:10.232729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-ef3f01a7-06a8-4e50-ac3f-8b44dbda9d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:10.321224 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1323/2658] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:10.327297 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afd1ae-efdd-45df-8427-23484174610c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:10.398389 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afd1ae-efdd-45df-8427-23484174610c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:10.403483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0846f6-6c71-4653-b747-c0348923ba97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:10.404160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0846f6-6c71-4653-b747-c0348923ba97 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.404451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0846f6-6c71-4653-b747-c0348923ba97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.404766 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0846f6-6c71-4653-b747-c0348923ba97 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.405477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0846f6-6c71-4653-b747-c0348923ba97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:10.407930 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0846f6-6c71-4653-b747-c0348923ba97 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:10.435186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1336/2659] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:58:10 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:58:10.439174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f01a7-06a8-4e50-ac3f-8b44dbda9d36 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:10.449914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f01a7-06a8-4e50-ac3f-8b44dbda9d36 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:10.506452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b828a-8741-4299-8cec-097c4af13eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:10.507122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b828a-8741-4299-8cec-097c4af13eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.507402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b828a-8741-4299-8cec-097c4af13eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.507676 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b828a-8741-4299-8cec-097c4af13eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.508381 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b828a-8741-4299-8cec-097c4af13eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:10.510326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b828a-8741-4299-8cec-097c4af13eb9 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:10.532663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1324/2660] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:58:10 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:58:10.536600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0846f6-6c71-4653-b747-c0348923ba97 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:10.547606 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0846f6-6c71-4653-b747-c0348923ba97 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:10.552854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a1a59-8f7f-40c7-9774-5f4e700714ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:10.553634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a1a59-8f7f-40c7-9774-5f4e700714ec admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.553917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a1a59-8f7f-40c7-9774-5f4e700714ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.554191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a1a59-8f7f-40c7-9774-5f4e700714ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.554919 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a1a59-8f7f-40c7-9774-5f4e700714ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:10.556501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a1a59-8f7f-40c7-9774-5f4e700714ec admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:10.567170 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1337/2661] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:10.571378 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b828a-8741-4299-8cec-097c4af13eb9 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:10.582121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b828a-8741-4299-8cec-097c4af13eb9 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:10.586804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29619395-b50a-4733-ae9c-a1e256cbebc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:10.587685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29619395-b50a-4733-ae9c-a1e256cbebc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.587972 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29619395-b50a-4733-ae9c-a1e256cbebc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.588252 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29619395-b50a-4733-ae9c-a1e256cbebc4 admin admin] PATH_INFO: `/v3/projects/0348beb1b2614fa1a3f1ece6e8c7473e/users/f90357edb11b4cbfa5dd23278eee2a6d/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.603535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29619395-b50a-4733-ae9c-a1e256cbebc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0348beb1b2614fa1a3f1ece6e8c7473e, user_id=f90357edb11b4cbfa5dd23278eee2a6d, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:10.606241 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29619395-b50a-4733-ae9c-a1e256cbebc4 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:10.618999 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1325/2662] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:58:10 2022] PUT /identity/v3/projects/0348beb1b2614fa1a3f1ece6e8c7473e/users/f90357edb11b4cbfa5dd23278eee2a6d/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:10.623218 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a1a59-8f7f-40c7-9774-5f4e700714ec admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:10.633215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a1a59-8f7f-40c7-9774-5f4e700714ec admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:10.689268 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a3568-ebde-4b55-ac2a-01b94890c2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:10.689858 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a3568-ebde-4b55-ac2a-01b94890c2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.690144 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a3568-ebde-4b55-ac2a-01b94890c2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.690421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a3568-ebde-4b55-ac2a-01b94890c2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.691159 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a3568-ebde-4b55-ac2a-01b94890c2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:10.692791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a3568-ebde-4b55-ac2a-01b94890c2a2 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:10.702704 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1338/2663] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:10.707208 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29619395-b50a-4733-ae9c-a1e256cbebc4 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:10.717578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29619395-b50a-4733-ae9c-a1e256cbebc4 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:10.723017 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ce2b5-d328-486d-8344-2b831a70da24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:10.723904 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ce2b5-d328-486d-8344-2b831a70da24 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.724190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ce2b5-d328-486d-8344-2b831a70da24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.724473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ce2b5-d328-486d-8344-2b831a70da24 admin admin] PATH_INFO: `/v3/projects/0348beb1b2614fa1a3f1ece6e8c7473e/users/f90357edb11b4cbfa5dd23278eee2a6d/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.726550 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ce2b5-d328-486d-8344-2b831a70da24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0348beb1b2614fa1a3f1ece6e8c7473e, user_id=f90357edb11b4cbfa5dd23278eee2a6d, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:10.729625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ce2b5-d328-486d-8344-2b831a70da24 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:10.739839 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1326/2664] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:58:10 2022] PUT /identity/v3/projects/0348beb1b2614fa1a3f1ece6e8c7473e/users/f90357edb11b4cbfa5dd23278eee2a6d/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:10.750565 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac067e-b584-437f-8306-86f4ae4e6261 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.750863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac067e-b584-437f-8306-86f4ae4e6261 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.751149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac067e-b584-437f-8306-86f4ae4e6261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.779493 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d7ac067e-b584-437f-8306-86f4ae4e6261 None None] MFA Rules not processed for user `f90357edb11b4cbfa5dd23278eee2a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:10.780743 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d7ac067e-b584-437f-8306-86f4ae4e6261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:10.834686 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:10.857546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1339/2665] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:10.868301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ce2b5-d328-486d-8344-2b831a70da24 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:10.870284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-418ce2b5-d328-486d-8344-2b831a70da24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:10.928777 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ce2b5-d328-486d-8344-2b831a70da24 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:10.932454 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246581e-c4df-4cf1-8278-021b315ee82a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:10.933130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246581e-c4df-4cf1-8278-021b315ee82a service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:10.933414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246581e-c4df-4cf1-8278-021b315ee82a service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:10.933692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246581e-c4df-4cf1-8278-021b315ee82a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:10.938353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:10.995586 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246581e-c4df-4cf1-8278-021b315ee82a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:10.996983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246581e-c4df-4cf1-8278-021b315ee82a service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:11.006617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1327/2666] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:58:15.431508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac067e-b584-437f-8306-86f4ae4e6261 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:15.441826 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac067e-b584-437f-8306-86f4ae4e6261 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:15.445496 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25f7ff-1f7e-4683-83a8-602d2fd43a66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:15.446142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25f7ff-1f7e-4683-83a8-602d2fd43a66 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:15.446426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25f7ff-1f7e-4683-83a8-602d2fd43a66 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:15.446704 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25f7ff-1f7e-4683-83a8-602d2fd43a66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:15.456396 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25f7ff-1f7e-4683-83a8-602d2fd43a66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:15.457869 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25f7ff-1f7e-4683-83a8-602d2fd43a66 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:15.467526 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1340/2667] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:58:17.715875 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246581e-c4df-4cf1-8278-021b315ee82a service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:17.717286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0246581e-c4df-4cf1-8278-021b315ee82a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:17.787976 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246581e-c4df-4cf1-8278-021b315ee82a service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:17.795086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677aaac9-db55-431b-8ae9-72d04fcb01fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:17.795086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677aaac9-db55-431b-8ae9-72d04fcb01fb service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:17.795086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677aaac9-db55-431b-8ae9-72d04fcb01fb service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:17.795086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677aaac9-db55-431b-8ae9-72d04fcb01fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:17.796080 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-677aaac9-db55-431b-8ae9-72d04fcb01fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:17.873453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677aaac9-db55-431b-8ae9-72d04fcb01fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:17.874902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677aaac9-db55-431b-8ae9-72d04fcb01fb service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:17.885135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1328/2668] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:58:20.420399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25f7ff-1f7e-4683-83a8-602d2fd43a66 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:20.487210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25f7ff-1f7e-4683-83a8-602d2fd43a66 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:20.491968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:20.492904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:20.493187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:20.493465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] PATH_INFO: `/v3/users/2c570307fc5647159324bf3794201976` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:20.494933 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c570307fc5647159324bf3794201976)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:20.496935 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:20.548927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c570307fc5647159324bf3794201976'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:20.548927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c570307fc5647159324bf3794201976'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:20.557399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c570307fc5647159324bf3794201976'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:20.573456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c570307fc5647159324bf3794201976'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:20.581889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c570307fc5647159324bf3794201976'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:20.588408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1341/2669] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:58:20 2022] DELETE /identity/v3/users/2c570307fc5647159324bf3794201976 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:20.593128 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286ff6-fdf0-415a-bd24-84d7eef79ab2 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:20.593597 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286ff6-fdf0-415a-bd24-84d7eef79ab2 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:20.593709 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286ff6-fdf0-415a-bd24-84d7eef79ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:20.620145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-34286ff6-fdf0-415a-bd24-84d7eef79ab2 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:20.621416 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-34286ff6-fdf0-415a-bd24-84d7eef79ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:20.675938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:20.688532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1329/2670] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:20.698237 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:20.699750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:20.769345 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4555580-a013-436d-9c0e-37c4a52a048e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:20.773615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992d78c-d0ef-4985-9818-6c2fee7f3c8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:20.774452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992d78c-d0ef-4985-9818-6c2fee7f3c8f service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:20.774887 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992d78c-d0ef-4985-9818-6c2fee7f3c8f service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:20.775262 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992d78c-d0ef-4985-9818-6c2fee7f3c8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:20.796152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:20.800811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992d78c-d0ef-4985-9818-6c2fee7f3c8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:20.802972 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992d78c-d0ef-4985-9818-6c2fee7f3c8f service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:20.818548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1342/2671] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:58:21.119221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34286ff6-fdf0-415a-bd24-84d7eef79ab2 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:21.121914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-34286ff6-fdf0-415a-bd24-84d7eef79ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:21.138205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34286ff6-fdf0-415a-bd24-84d7eef79ab2 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:21.143469 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:21.144353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:21.144923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:21.145210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] PATH_INFO: `/v3/projects/18245c11e5f549d083d1adc4d3fe4d9c` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:21.146433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18245c11e5f549d083d1adc4d3fe4d9c)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:21.148351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:21.198367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18245c11e5f549d083d1adc4d3fe4d9c', 'request_id': 'req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.207413 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18245c11e5f549d083d1adc4d3fe4d9c', 'request_id': 'req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.213804 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18245c11e5f549d083d1adc4d3fe4d9c', 'request_id': 'req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.226376 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] The token cache is being invalidate because project 18245c11e5f549d083d1adc4d3fe4d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:58:21.226665 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.227324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.228933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1330/2672] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:58:21 2022] DELETE /identity/v3/projects/18245c11e5f549d083d1adc4d3fe4d9c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:21.232953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992d78c-d0ef-4985-9818-6c2fee7f3c8f service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:21.243290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992d78c-d0ef-4985-9818-6c2fee7f3c8f service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:21.306577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14d8db-9848-4418-9928-457269ee8fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:21.308141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14d8db-9848-4418-9928-457269ee8fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:21.308141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14d8db-9848-4418-9928-457269ee8fac admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:21.308141 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14d8db-9848-4418-9928-457269ee8fac admin admin] PATH_INFO: `/v3/users/f90357edb11b4cbfa5dd23278eee2a6d` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:21.309954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14d8db-9848-4418-9928-457269ee8fac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f90357edb11b4cbfa5dd23278eee2a6d)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:21.311879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14d8db-9848-4418-9928-457269ee8fac admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:21.362116 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8c14d8db-9848-4418-9928-457269ee8fac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f90357edb11b4cbfa5dd23278eee2a6d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.362787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8c14d8db-9848-4418-9928-457269ee8fac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f90357edb11b4cbfa5dd23278eee2a6d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.370498 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8c14d8db-9848-4418-9928-457269ee8fac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f90357edb11b4cbfa5dd23278eee2a6d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.383408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8c14d8db-9848-4418-9928-457269ee8fac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f90357edb11b4cbfa5dd23278eee2a6d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.390453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-8c14d8db-9848-4418-9928-457269ee8fac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f90357edb11b4cbfa5dd23278eee2a6d'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.396946 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1343/2673] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:58:21 2022] DELETE /identity/v3/users/f90357edb11b4cbfa5dd23278eee2a6d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:21.680338 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:21.682728 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:21.698674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d38ea1-cf0e-41ac-a63e-b1982fa1bd79 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:21.762704 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:21.763577 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:21.763862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:21.764142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5 admin admin] PATH_INFO: `/v3/projects/0348beb1b2614fa1a3f1ece6e8c7473e` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:21.765381 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0348beb1b2614fa1a3f1ece6e8c7473e)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:21.767924 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:21.830694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0348beb1b2614fa1a3f1ece6e8c7473e', 'request_id': 'req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.840022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0348beb1b2614fa1a3f1ece6e8c7473e', 'request_id': 'req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.845952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0348beb1b2614fa1a3f1ece6e8c7473e', 'request_id': 'req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.860341 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5 admin admin] The token cache is being invalidate because project 0348beb1b2614fa1a3f1ece6e8c7473e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:58:21.860684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.861354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8b60d4b5-a2ab-4c8a-ab0f-b3f4bb47c3c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:21.862936 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1331/2674] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:58:21 2022] DELETE /identity/v3/projects/0348beb1b2614fa1a3f1ece6e8c7473e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:21.869240 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61f47a-f690-4e1c-b591-7c13b3ed7705 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:21.870052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61f47a-f690-4e1c-b591-7c13b3ed7705 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:21.870325 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61f47a-f690-4e1c-b591-7c13b3ed7705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:21.891678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4e61f47a-f690-4e1c-b591-7c13b3ed7705 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:21.892953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4e61f47a-f690-4e1c-b591-7c13b3ed7705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:21.957448 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1344/2675] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:58:21.966274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f85cc-cb61-42be-8b35-028106cd6d2f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:21.966568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f85cc-cb61-42be-8b35-028106cd6d2f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:21.966856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f85cc-cb61-42be-8b35-028106cd6d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:21.989513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-445f85cc-cb61-42be-8b35-028106cd6d2f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:21.991917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-445f85cc-cb61-42be-8b35-028106cd6d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:22.002124 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1332/2676] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:22.006781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61f47a-f690-4e1c-b591-7c13b3ed7705 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:22.016951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61f47a-f690-4e1c-b591-7c13b3ed7705 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:22.021734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c17d21-35dc-4207-aff0-fe4e5127013f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:22.022407 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c17d21-35dc-4207-aff0-fe4e5127013f admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:22.022700 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c17d21-35dc-4207-aff0-fe4e5127013f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:22.022984 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c17d21-35dc-4207-aff0-fe4e5127013f admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:22.023720 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c17d21-35dc-4207-aff0-fe4e5127013f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:58:22.024035 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c17d21-35dc-4207-aff0-fe4e5127013f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:22.025670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c17d21-35dc-4207-aff0-fe4e5127013f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:22.037909 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1345/2677] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 08:58:22.042815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f85cc-cb61-42be-8b35-028106cd6d2f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:22.048087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:22.054558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f85cc-cb61-42be-8b35-028106cd6d2f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:22.060497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb487794-af66-4036-8f0d-cde87ebbc442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:22.061214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb487794-af66-4036-8f0d-cde87ebbc442 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:22.061498 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb487794-af66-4036-8f0d-cde87ebbc442 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:22.061775 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb487794-af66-4036-8f0d-cde87ebbc442 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:22.062487 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb487794-af66-4036-8f0d-cde87ebbc442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:22.065070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb487794-af66-4036-8f0d-cde87ebbc442 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:22.093304 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1333/2678] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:58:22 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:58:22.097615 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c17d21-35dc-4207-aff0-fe4e5127013f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:22.102860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:22.109666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c17d21-35dc-4207-aff0-fe4e5127013f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:22.170255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271484f-f482-473a-84e0-6032e21c89f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:22.170942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271484f-f482-473a-84e0-6032e21c89f7 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:22.171227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271484f-f482-473a-84e0-6032e21c89f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:22.171507 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271484f-f482-473a-84e0-6032e21c89f7 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:22.172225 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271484f-f482-473a-84e0-6032e21c89f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:22.174798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271484f-f482-473a-84e0-6032e21c89f7 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:22.197934 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1346/2679] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:58:22 2022] POST /identity/v3/users => generated 280 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:58:22.202202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb487794-af66-4036-8f0d-cde87ebbc442 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:22.214486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb487794-af66-4036-8f0d-cde87ebbc442 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:22.220685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272d5d5-6793-4980-a8f1-59309fecd39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:22.221386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272d5d5-6793-4980-a8f1-59309fecd39a admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:22.221671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272d5d5-6793-4980-a8f1-59309fecd39a admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:22.221950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272d5d5-6793-4980-a8f1-59309fecd39a admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:22.222731 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272d5d5-6793-4980-a8f1-59309fecd39a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:22.224367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272d5d5-6793-4980-a8f1-59309fecd39a admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:22.243262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1334/2680] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:22.247678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271484f-f482-473a-84e0-6032e21c89f7 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:22.260805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271484f-f482-473a-84e0-6032e21c89f7 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:22.262212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272d5d5-6793-4980-a8f1-59309fecd39a admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:22.264333 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3272d5d5-6793-4980-a8f1-59309fecd39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:22.266309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6344e2-7a01-4a6e-b454-1a263f32114e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:22.267091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6344e2-7a01-4a6e-b454-1a263f32114e admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:22.267477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6344e2-7a01-4a6e-b454-1a263f32114e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:22.267861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6344e2-7a01-4a6e-b454-1a263f32114e admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:22.268787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6344e2-7a01-4a6e-b454-1a263f32114e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:22.270751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6344e2-7a01-4a6e-b454-1a263f32114e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:22.290379 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1347/2681] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:22.292441 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6344e2-7a01-4a6e-b454-1a263f32114e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:22.312282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6344e2-7a01-4a6e-b454-1a263f32114e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:22.317721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533af2f-7a47-4c69-b7d3-829f99eaa848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:22.318632 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533af2f-7a47-4c69-b7d3-829f99eaa848 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:22.319071 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533af2f-7a47-4c69-b7d3-829f99eaa848 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:22.319358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533af2f-7a47-4c69-b7d3-829f99eaa848 admin admin] PATH_INFO: `/v3/projects/436a6621708d42de9a5f177433855553/users/8715226feb434a0d8c55b37f4d562d8c/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:22.337158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533af2f-7a47-4c69-b7d3-829f99eaa848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=436a6621708d42de9a5f177433855553, user_id=8715226feb434a0d8c55b37f4d562d8c, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:22.340255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533af2f-7a47-4c69-b7d3-829f99eaa848 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:22.351211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272d5d5-6793-4980-a8f1-59309fecd39a admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:22.356843 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1348/2682] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:58:22 2022] PUT /identity/v3/projects/436a6621708d42de9a5f177433855553/users/8715226feb434a0d8c55b37f4d562d8c/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:22.413422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b93f72-956d-481b-ac18-9f8d4a00b2ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:22.414094 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b93f72-956d-481b-ac18-9f8d4a00b2ed service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:22.414369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b93f72-956d-481b-ac18-9f8d4a00b2ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:22.414645 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b93f72-956d-481b-ac18-9f8d4a00b2ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:22.416710 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-36b93f72-956d-481b-ac18-9f8d4a00b2ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:22.429404 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfcebf-e3a1-4906-95f8-c6799f574a6e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:22.430142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfcebf-e3a1-4906-95f8-c6799f574a6e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:22.430602 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfcebf-e3a1-4906-95f8-c6799f574a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:22.468299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-8dbfcebf-e3a1-4906-95f8-c6799f574a6e None None] MFA Rules not processed for user `8715226feb434a0d8c55b37f4d562d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:22.469634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-8dbfcebf-e3a1-4906-95f8-c6799f574a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:22.505427 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b93f72-956d-481b-ac18-9f8d4a00b2ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:22.506893 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b93f72-956d-481b-ac18-9f8d4a00b2ed service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:22.532767 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1335/2683] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:58:22.554189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1349/2684] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:22 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:58:22.588606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b93f72-956d-481b-ac18-9f8d4a00b2ed service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:22.590318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-36b93f72-956d-481b-ac18-9f8d4a00b2ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:22.602095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b93f72-956d-481b-ac18-9f8d4a00b2ed service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:22.605769 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4a61c-9190-468d-9b26-a2d8df019ade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:22.606425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4a61c-9190-468d-9b26-a2d8df019ade service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:22.606711 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4a61c-9190-468d-9b26-a2d8df019ade service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:22.607022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4a61c-9190-468d-9b26-a2d8df019ade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:22.616780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4a61c-9190-468d-9b26-a2d8df019ade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:22.618227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4a61c-9190-468d-9b26-a2d8df019ade service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:22.631942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1336/2685] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:22 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:58:22.648087 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbfcebf-e3a1-4906-95f8-c6799f574a6e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:22.696678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:22.716001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbfcebf-e3a1-4906-95f8-c6799f574a6e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:22.720772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:22.721651 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:22.722111 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:22.722391 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] PATH_INFO: `/v3/users/8715226feb434a0d8c55b37f4d562d8c` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:22.723607 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8715226feb434a0d8c55b37f4d562d8c)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:22.725575 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:22.775452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8715226feb434a0d8c55b37f4d562d8c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:22.776140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8715226feb434a0d8c55b37f4d562d8c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:22.784052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8715226feb434a0d8c55b37f4d562d8c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:22.796941 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8715226feb434a0d8c55b37f4d562d8c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:22.804783 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8715226feb434a0d8c55b37f4d562d8c'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:22.811154 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1350/2686] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:58:22 2022] DELETE /identity/v3/users/8715226feb434a0d8c55b37f4d562d8c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:22.816271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c2af3-b0a1-46c6-96ac-c40bce637fdb None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:22.816796 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c2af3-b0a1-46c6-96ac-c40bce637fdb None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:22.816915 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c2af3-b0a1-46c6-96ac-c40bce637fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:22.836341 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-5c3c2af3-b0a1-46c6-96ac-c40bce637fdb None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:22.837605 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5c3c2af3-b0a1-46c6-96ac-c40bce637fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:22.850209 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:22.902647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1337/2687] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:22.912937 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:22.914349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:22.972212 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e75a04-6c42-41a5-8475-a822c6139de1 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:22.975952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84a2ab-7c96-45da-af90-2d7448314634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:22.976633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84a2ab-7c96-45da-af90-2d7448314634 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:22.976922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84a2ab-7c96-45da-af90-2d7448314634 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:22.977202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84a2ab-7c96-45da-af90-2d7448314634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:22.987600 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84a2ab-7c96-45da-af90-2d7448314634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:22.989079 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84a2ab-7c96-45da-af90-2d7448314634 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:22.999131 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1351/2688] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:58:23.282205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c2af3-b0a1-46c6-96ac-c40bce637fdb None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:23.283971 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-5c3c2af3-b0a1-46c6-96ac-c40bce637fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:23.304988 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c2af3-b0a1-46c6-96ac-c40bce637fdb None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:23.311086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afb3e0-c7de-4fb4-b5d2-171af813c23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:23.311874 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afb3e0-c7de-4fb4-b5d2-171af813c23d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:23.312148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afb3e0-c7de-4fb4-b5d2-171af813c23d admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:23.312455 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afb3e0-c7de-4fb4-b5d2-171af813c23d admin admin] PATH_INFO: `/v3/projects/436a6621708d42de9a5f177433855553` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:23.313688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34afb3e0-c7de-4fb4-b5d2-171af813c23d admin admin] RBAC: Authorizing `identity:delete_project(project_id=436a6621708d42de9a5f177433855553)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:23.315646 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34afb3e0-c7de-4fb4-b5d2-171af813c23d admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:23.364447 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-34afb3e0-c7de-4fb4-b5d2-171af813c23d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '436a6621708d42de9a5f177433855553', 'request_id': 'req-34afb3e0-c7de-4fb4-b5d2-171af813c23d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:23.372463 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-34afb3e0-c7de-4fb4-b5d2-171af813c23d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '436a6621708d42de9a5f177433855553', 'request_id': 'req-34afb3e0-c7de-4fb4-b5d2-171af813c23d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:23.378327 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-34afb3e0-c7de-4fb4-b5d2-171af813c23d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '436a6621708d42de9a5f177433855553', 'request_id': 'req-34afb3e0-c7de-4fb4-b5d2-171af813c23d'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:23.389721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-34afb3e0-c7de-4fb4-b5d2-171af813c23d admin admin] The token cache is being invalidate because project 436a6621708d42de9a5f177433855553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:58:23.390015 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-34afb3e0-c7de-4fb4-b5d2-171af813c23d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:23.390680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-34afb3e0-c7de-4fb4-b5d2-171af813c23d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:23.392781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1338/2689] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:58:23 2022] DELETE /identity/v3/projects/436a6621708d42de9a5f177433855553 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:23.398610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80db9c2-528d-45f3-bf2e-4404d4e7248c None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:23.398908 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80db9c2-528d-45f3-bf2e-4404d4e7248c None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:23.399222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80db9c2-528d-45f3-bf2e-4404d4e7248c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:23.419669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-e80db9c2-528d-45f3-bf2e-4404d4e7248c None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:23.420866 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e80db9c2-528d-45f3-bf2e-4404d4e7248c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:23.482666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1352/2690] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:23.490253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0c4b1-b2d2-4b30-be2a-1d72e10ec1fb None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:23.490549 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0c4b1-b2d2-4b30-be2a-1d72e10ec1fb None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:23.490839 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0c4b1-b2d2-4b30-be2a-1d72e10ec1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:23.510827 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-8fe0c4b1-b2d2-4b30-be2a-1d72e10ec1fb None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:23.512043 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8fe0c4b1-b2d2-4b30-be2a-1d72e10ec1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:23.520668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1339/2691] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:23.525048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80db9c2-528d-45f3-bf2e-4404d4e7248c None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:23.536045 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80db9c2-528d-45f3-bf2e-4404d4e7248c None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:23.540812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347f87b-e2be-495c-8b87-f5d442a4bc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:23.541484 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347f87b-e2be-495c-8b87-f5d442a4bc16 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:23.541768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347f87b-e2be-495c-8b87-f5d442a4bc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:23.542046 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347f87b-e2be-495c-8b87-f5d442a4bc16 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:23.542817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347f87b-e2be-495c-8b87-f5d442a4bc16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:58:23.543127 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347f87b-e2be-495c-8b87-f5d442a4bc16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:23.544759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347f87b-e2be-495c-8b87-f5d442a4bc16 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:23.547497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:23.559557 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1353/2692] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:58:23.564667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0c4b1-b2d2-4b30-be2a-1d72e10ec1fb None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:23.574038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0c4b1-b2d2-4b30-be2a-1d72e10ec1fb None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:23.579396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af9163-0012-4230-aab0-a96db5c74194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:23.580055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af9163-0012-4230-aab0-a96db5c74194 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:23.580343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af9163-0012-4230-aab0-a96db5c74194 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:23.580866 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af9163-0012-4230-aab0-a96db5c74194 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:23.581362 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af9163-0012-4230-aab0-a96db5c74194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:23.583253 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af9163-0012-4230-aab0-a96db5c74194 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:23.615902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1340/2693] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:58:23 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:58:23.617123 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347f87b-e2be-495c-8b87-f5d442a4bc16 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:23.628324 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347f87b-e2be-495c-8b87-f5d442a4bc16 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:23.690417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb10c1d-33d9-49f9-a4b5-6be1daf93a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:23.690988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb10c1d-33d9-49f9-a4b5-6be1daf93a97 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:23.691274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb10c1d-33d9-49f9-a4b5-6be1daf93a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:23.691553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb10c1d-33d9-49f9-a4b5-6be1daf93a97 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:23.692263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb10c1d-33d9-49f9-a4b5-6be1daf93a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:23.694261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb10c1d-33d9-49f9-a4b5-6be1daf93a97 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:23.727477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1354/2694] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:58:23 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:58:23.731825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af9163-0012-4230-aab0-a96db5c74194 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:23.742954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af9163-0012-4230-aab0-a96db5c74194 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:23.754260 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78595d-ed12-4dce-a9e9-905ba4f3ead7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:23.755036 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78595d-ed12-4dce-a9e9-905ba4f3ead7 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:23.755384 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78595d-ed12-4dce-a9e9-905ba4f3ead7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:23.755727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78595d-ed12-4dce-a9e9-905ba4f3ead7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:23.756589 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78595d-ed12-4dce-a9e9-905ba4f3ead7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:23.758246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78595d-ed12-4dce-a9e9-905ba4f3ead7 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:23.769467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1341/2695] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:23.773769 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb10c1d-33d9-49f9-a4b5-6be1daf93a97 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:23.783983 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb10c1d-33d9-49f9-a4b5-6be1daf93a97 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:23.789052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e035d3-2755-460c-99e5-fc9509045606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:23.789723 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e035d3-2755-460c-99e5-fc9509045606 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:23.790010 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e035d3-2755-460c-99e5-fc9509045606 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:23.790291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e035d3-2755-460c-99e5-fc9509045606 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:23.791027 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e035d3-2755-460c-99e5-fc9509045606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:23.792657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e035d3-2755-460c-99e5-fc9509045606 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:23.803185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1355/2696] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:23.807414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78595d-ed12-4dce-a9e9-905ba4f3ead7 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:23.817931 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78595d-ed12-4dce-a9e9-905ba4f3ead7 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:23.822587 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3945547-c9c4-479e-b6d1-d0f5e5844d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:23.823465 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3945547-c9c4-479e-b6d1-d0f5e5844d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:23.823748 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3945547-c9c4-479e-b6d1-d0f5e5844d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:23.824021 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3945547-c9c4-479e-b6d1-d0f5e5844d66 admin admin] PATH_INFO: `/v3/projects/c61846255f854229baeeba0b72eaa477/users/811d782c18e44b6c88381978a86f53ca/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:23.839850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3945547-c9c4-479e-b6d1-d0f5e5844d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61846255f854229baeeba0b72eaa477, user_id=811d782c18e44b6c88381978a86f53ca, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:23.842520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3945547-c9c4-479e-b6d1-d0f5e5844d66 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:23.856041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1342/2697] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:58:23 2022] PUT /identity/v3/projects/c61846255f854229baeeba0b72eaa477/users/811d782c18e44b6c88381978a86f53ca/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:23.866456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0aa88-9b20-4aec-a5eb-93e02d4b4079 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:23.866748 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0aa88-9b20-4aec-a5eb-93e02d4b4079 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:23.867028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0aa88-9b20-4aec-a5eb-93e02d4b4079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:23.898539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-f9a0aa88-9b20-4aec-a5eb-93e02d4b4079 None None] MFA Rules not processed for user `811d782c18e44b6c88381978a86f53ca`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:23.898539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f9a0aa88-9b20-4aec-a5eb-93e02d4b4079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:23.962620 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1356/2698] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:23.974426 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3945547-c9c4-479e-b6d1-d0f5e5844d66 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:23.976100 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e3945547-c9c4-479e-b6d1-d0f5e5844d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:24.018182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:24.034888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3945547-c9c4-479e-b6d1-d0f5e5844d66 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:24.038548 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93206003-b992-4e43-a34a-d3e998c1861f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '925bf2da4df74e86aa84ec62a267d088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:24.039195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93206003-b992-4e43-a34a-d3e998c1861f service glance] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:24.039481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93206003-b992-4e43-a34a-d3e998c1861f service glance] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:24.039757 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93206003-b992-4e43-a34a-d3e998c1861f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:24.048886 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93206003-b992-4e43-a34a-d3e998c1861f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:24.050328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93206003-b992-4e43-a34a-d3e998c1861f service glance] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:24.059232 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1343/2699] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:23 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:58:33.959651 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:34.055168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=9 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:35.558532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0aa88-9b20-4aec-a5eb-93e02d4b4079 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:35.559927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f9a0aa88-9b20-4aec-a5eb-93e02d4b4079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:35.617680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0aa88-9b20-4aec-a5eb-93e02d4b4079 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:35.621561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d9163-b08f-440c-8d18-f8fe0a2c4a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:35.622214 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d9163-b08f-440c-8d18-f8fe0a2c4a32 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:35.622497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d9163-b08f-440c-8d18-f8fe0a2c4a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:35.622777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d9163-b08f-440c-8d18-f8fe0a2c4a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:35.624606 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-788d9163-b08f-440c-8d18-f8fe0a2c4a32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:35.697827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d9163-b08f-440c-8d18-f8fe0a2c4a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:35.699501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d9163-b08f-440c-8d18-f8fe0a2c4a32 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:35.710780 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1357/2700] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:58:38.980662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93206003-b992-4e43-a34a-d3e998c1861f service glance] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:38.982312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-93206003-b992-4e43-a34a-d3e998c1861f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:38.998746 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93206003-b992-4e43-a34a-d3e998c1861f service glance] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:39.003575 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780543c7-8c3f-4be5-ba03-8efe99f5bbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:39.004492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780543c7-8c3f-4be5-ba03-8efe99f5bbda admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:39.004815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780543c7-8c3f-4be5-ba03-8efe99f5bbda admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:39.005099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780543c7-8c3f-4be5-ba03-8efe99f5bbda admin admin] PATH_INFO: `/v3/users/8e682232b7374450a7265e092602a5a6` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:39.006329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780543c7-8c3f-4be5-ba03-8efe99f5bbda admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e682232b7374450a7265e092602a5a6)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:39.008266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780543c7-8c3f-4be5-ba03-8efe99f5bbda admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:39.058386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-780543c7-8c3f-4be5-ba03-8efe99f5bbda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e682232b7374450a7265e092602a5a6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:39.066326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-780543c7-8c3f-4be5-ba03-8efe99f5bbda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e682232b7374450a7265e092602a5a6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:39.078585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-780543c7-8c3f-4be5-ba03-8efe99f5bbda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e682232b7374450a7265e092602a5a6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:39.086374 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-780543c7-8c3f-4be5-ba03-8efe99f5bbda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e682232b7374450a7265e092602a5a6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:39.087445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-780543c7-8c3f-4be5-ba03-8efe99f5bbda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e682232b7374450a7265e092602a5a6'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:39.095318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1344/2701] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:58:38 2022] DELETE /identity/v3/users/8e682232b7374450a7265e092602a5a6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:39.276362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d9163-b08f-440c-8d18-f8fe0a2c4a32 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:39.287183 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d9163-b08f-440c-8d18-f8fe0a2c4a32 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:39.381293 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f5111-aae9-49a1-b8ac-c5a0380d1838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:39.382243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f5111-aae9-49a1-b8ac-c5a0380d1838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:39.382528 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f5111-aae9-49a1-b8ac-c5a0380d1838 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:39.382806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f5111-aae9-49a1-b8ac-c5a0380d1838 admin admin] PATH_INFO: `/v3/projects/86bea90252bc44b580c40b24b0bd4ed3` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:39.384049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f5111-aae9-49a1-b8ac-c5a0380d1838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86bea90252bc44b580c40b24b0bd4ed3)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:39.386026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f5111-aae9-49a1-b8ac-c5a0380d1838 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:39.442517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-632f5111-aae9-49a1-b8ac-c5a0380d1838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86bea90252bc44b580c40b24b0bd4ed3', 'request_id': 'req-632f5111-aae9-49a1-b8ac-c5a0380d1838'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:39.444830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:39.450473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-632f5111-aae9-49a1-b8ac-c5a0380d1838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86bea90252bc44b580c40b24b0bd4ed3', 'request_id': 'req-632f5111-aae9-49a1-b8ac-c5a0380d1838'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:39.461764 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-632f5111-aae9-49a1-b8ac-c5a0380d1838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86bea90252bc44b580c40b24b0bd4ed3', 'request_id': 'req-632f5111-aae9-49a1-b8ac-c5a0380d1838'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:39.473544 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-632f5111-aae9-49a1-b8ac-c5a0380d1838 admin admin] The token cache is being invalidate because project 86bea90252bc44b580c40b24b0bd4ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:58:39.473831 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-632f5111-aae9-49a1-b8ac-c5a0380d1838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:39.474532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-632f5111-aae9-49a1-b8ac-c5a0380d1838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:39.476107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1358/2702] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:58:39 2022] DELETE /identity/v3/projects/86bea90252bc44b580c40b24b0bd4ed3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:39.482161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7e03f-41cb-4f3c-8767-7ad45aee3e4f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:39.482679 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7e03f-41cb-4f3c-8767-7ad45aee3e4f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:39.483183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7e03f-41cb-4f3c-8767-7ad45aee3e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:39.506312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-d8e7e03f-41cb-4f3c-8767-7ad45aee3e4f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:39.507575 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d8e7e03f-41cb-4f3c-8767-7ad45aee3e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:39.571164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1345/2703] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:39.579597 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc645156-e7dc-4e8e-a3f4-3d6b730a46b0 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:39.579892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc645156-e7dc-4e8e-a3f4-3d6b730a46b0 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:39.580172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc645156-e7dc-4e8e-a3f4-3d6b730a46b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:39.603010 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-cc645156-e7dc-4e8e-a3f4-3d6b730a46b0 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:39.604239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-cc645156-e7dc-4e8e-a3f4-3d6b730a46b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:39.613048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1359/2704] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:39.617616 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7e03f-41cb-4f3c-8767-7ad45aee3e4f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:39.628113 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7e03f-41cb-4f3c-8767-7ad45aee3e4f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:39.632690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c4ef8-16c1-47dc-aede-42c9b2c0ecfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:39.633389 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c4ef8-16c1-47dc-aede-42c9b2c0ecfd admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:39.633675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c4ef8-16c1-47dc-aede-42c9b2c0ecfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:39.633950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c4ef8-16c1-47dc-aede-42c9b2c0ecfd admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:39.634685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c4ef8-16c1-47dc-aede-42c9b2c0ecfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:58:39.634993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c4ef8-16c1-47dc-aede-42c9b2c0ecfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:39.636619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c4ef8-16c1-47dc-aede-42c9b2c0ecfd admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:39.649210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1346/2705] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:58:39.653512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc645156-e7dc-4e8e-a3f4-3d6b730a46b0 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:39.663186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc645156-e7dc-4e8e-a3f4-3d6b730a46b0 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:39.667940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58713ba-331e-490b-a723-3bb975345331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:39.668628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58713ba-331e-490b-a723-3bb975345331 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:39.668920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58713ba-331e-490b-a723-3bb975345331 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:39.669200 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58713ba-331e-490b-a723-3bb975345331 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:39.669909 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58713ba-331e-490b-a723-3bb975345331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:39.671836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58713ba-331e-490b-a723-3bb975345331 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:39.700623 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1360/2706] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:58:39 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:58:39.705568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c4ef8-16c1-47dc-aede-42c9b2c0ecfd admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:39.715859 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c4ef8-16c1-47dc-aede-42c9b2c0ecfd admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:39.773348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f7b40-9a82-43f1-aba3-7b46aae2a7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:39.774025 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f7b40-9a82-43f1-aba3-7b46aae2a7ec admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:39.774306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f7b40-9a82-43f1-aba3-7b46aae2a7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:39.774575 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f7b40-9a82-43f1-aba3-7b46aae2a7ec admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:39.775264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f7b40-9a82-43f1-aba3-7b46aae2a7ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:39.777214 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f7b40-9a82-43f1-aba3-7b46aae2a7ec admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:39.785430 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:39.800505 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1347/2707] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:58:39 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:58:39.804747 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58713ba-331e-490b-a723-3bb975345331 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:39.815929 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58713ba-331e-490b-a723-3bb975345331 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:39.820678 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5ed6b-4a30-4344-ba0d-d482f6d80e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:39.821346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5ed6b-4a30-4344-ba0d-d482f6d80e92 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:39.821631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5ed6b-4a30-4344-ba0d-d482f6d80e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:39.821909 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5ed6b-4a30-4344-ba0d-d482f6d80e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:39.822662 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5ed6b-4a30-4344-ba0d-d482f6d80e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:39.824253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5ed6b-4a30-4344-ba0d-d482f6d80e92 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:39.835202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1361/2708] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:39.839195 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f7b40-9a82-43f1-aba3-7b46aae2a7ec admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:39.849559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f7b40-9a82-43f1-aba3-7b46aae2a7ec admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:39.854328 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e45684-8743-49e7-806d-f4234fbaa9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:39.855002 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e45684-8743-49e7-806d-f4234fbaa9be admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:39.855283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e45684-8743-49e7-806d-f4234fbaa9be admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:39.855559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e45684-8743-49e7-806d-f4234fbaa9be admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:39.856291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e45684-8743-49e7-806d-f4234fbaa9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:39.857937 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e45684-8743-49e7-806d-f4234fbaa9be admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:39.869625 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1348/2709] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:39.875060 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5ed6b-4a30-4344-ba0d-d482f6d80e92 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:39.886042 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5ed6b-4a30-4344-ba0d-d482f6d80e92 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:39.891020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b3c80-3393-400c-b50d-11bc3ee4fee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:39.891914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b3c80-3393-400c-b50d-11bc3ee4fee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:39.892205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b3c80-3393-400c-b50d-11bc3ee4fee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:39.892487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b3c80-3393-400c-b50d-11bc3ee4fee6 admin admin] PATH_INFO: `/v3/projects/9084169d0b4a4e1986c30416b9413f33/users/ea33b7ac96ef47a2b1739e5a90353f82/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:39.909146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b3c80-3393-400c-b50d-11bc3ee4fee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9084169d0b4a4e1986c30416b9413f33, user_id=ea33b7ac96ef47a2b1739e5a90353f82, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:39.911893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b3c80-3393-400c-b50d-11bc3ee4fee6 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:39.925781 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1362/2710] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:58:39 2022] PUT /identity/v3/projects/9084169d0b4a4e1986c30416b9413f33/users/ea33b7ac96ef47a2b1739e5a90353f82/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:39.937437 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbce10d-fda6-4a60-abc5-164b074ff995 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:39.937727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbce10d-fda6-4a60-abc5-164b074ff995 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:39.938006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbce10d-fda6-4a60-abc5-164b074ff995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:39.967244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-9cbce10d-fda6-4a60-abc5-164b074ff995 None None] MFA Rules not processed for user `ea33b7ac96ef47a2b1739e5a90353f82`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:39.968368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-9cbce10d-fda6-4a60-abc5-164b074ff995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:40.045809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1349/2711] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:40.057957 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b3c80-3393-400c-b50d-11bc3ee4fee6 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:40.059730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-774b3c80-3393-400c-b50d-11bc3ee4fee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:40.134577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b3c80-3393-400c-b50d-11bc3ee4fee6 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:40.140377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d1e01-c61b-4b78-b21a-433a3d6f261b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:40.141066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d1e01-c61b-4b78-b21a-433a3d6f261b service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:40.141376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d1e01-c61b-4b78-b21a-433a3d6f261b service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:40.141656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d1e01-c61b-4b78-b21a-433a3d6f261b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:40.150280 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d1e01-c61b-4b78-b21a-433a3d6f261b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:40.151871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d1e01-c61b-4b78-b21a-433a3d6f261b service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:40.163910 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1363/2712] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:58:43.936326 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbce10d-fda6-4a60-abc5-164b074ff995 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:44.004095 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbce10d-fda6-4a60-abc5-164b074ff995 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:44.008789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:44.009672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:44.010011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:44.010370 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] PATH_INFO: `/v3/users/ea33b7ac96ef47a2b1739e5a90353f82` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:44.011599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea33b7ac96ef47a2b1739e5a90353f82)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:44.013595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:44.063841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea33b7ac96ef47a2b1739e5a90353f82'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:44.071600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea33b7ac96ef47a2b1739e5a90353f82'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:44.082758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:44.084112 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea33b7ac96ef47a2b1739e5a90353f82'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:44.093390 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea33b7ac96ef47a2b1739e5a90353f82'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:44.094552 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea33b7ac96ef47a2b1739e5a90353f82'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:44.101136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1350/2713] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:58:43 2022] DELETE /identity/v3/users/ea33b7ac96ef47a2b1739e5a90353f82 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:44.110140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bfd33-66f4-47d2-8752-a607e12dc110 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:44.111337 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bfd33-66f4-47d2-8752-a607e12dc110 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:44.111617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bfd33-66f4-47d2-8752-a607e12dc110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:44.131893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d05bfd33-66f4-47d2-8752-a607e12dc110 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:44.133149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d05bfd33-66f4-47d2-8752-a607e12dc110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:44.196949 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1364/2714] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:44.206841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:44.208268 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:44.266130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52adbc-d2c1-43a5-845e-fa9f72870def admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:44.270244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2020e-8a36-4f20-80da-508ea43aed36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:44.271006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2020e-8a36-4f20-80da-508ea43aed36 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:44.271286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2020e-8a36-4f20-80da-508ea43aed36 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:44.271562 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2020e-8a36-4f20-80da-508ea43aed36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:44.281264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2020e-8a36-4f20-80da-508ea43aed36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:44.283246 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2020e-8a36-4f20-80da-508ea43aed36 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:44.294046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1351/2715] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:58:44.587142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bfd33-66f4-47d2-8752-a607e12dc110 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:44.588643 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d05bfd33-66f4-47d2-8752-a607e12dc110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:44.597706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:44.621814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bfd33-66f4-47d2-8752-a607e12dc110 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:44.627118 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23854c0-bf5b-4143-9d0f-72037bc36092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:44.627991 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23854c0-bf5b-4143-9d0f-72037bc36092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:44.628274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23854c0-bf5b-4143-9d0f-72037bc36092 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:44.628553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23854c0-bf5b-4143-9d0f-72037bc36092 admin admin] PATH_INFO: `/v3/projects/9084169d0b4a4e1986c30416b9413f33` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:44.629792 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23854c0-bf5b-4143-9d0f-72037bc36092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9084169d0b4a4e1986c30416b9413f33)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:44.631768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23854c0-bf5b-4143-9d0f-72037bc36092 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:44.679841 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f23854c0-bf5b-4143-9d0f-72037bc36092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9084169d0b4a4e1986c30416b9413f33', 'request_id': 'req-f23854c0-bf5b-4143-9d0f-72037bc36092'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:44.687011 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f23854c0-bf5b-4143-9d0f-72037bc36092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9084169d0b4a4e1986c30416b9413f33', 'request_id': 'req-f23854c0-bf5b-4143-9d0f-72037bc36092'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:44.693255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f23854c0-bf5b-4143-9d0f-72037bc36092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9084169d0b4a4e1986c30416b9413f33', 'request_id': 'req-f23854c0-bf5b-4143-9d0f-72037bc36092'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:44.705350 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f23854c0-bf5b-4143-9d0f-72037bc36092 admin admin] The token cache is being invalidate because project 9084169d0b4a4e1986c30416b9413f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:58:44.705591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f23854c0-bf5b-4143-9d0f-72037bc36092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:44.706254 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f23854c0-bf5b-4143-9d0f-72037bc36092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:58:44.707800 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1365/2716] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:58:44 2022] DELETE /identity/v3/projects/9084169d0b4a4e1986c30416b9413f33 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:44.713834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663a430-1842-4f07-b7f6-30577dfeec8f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:44.716678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663a430-1842-4f07-b7f6-30577dfeec8f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:44.716678 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663a430-1842-4f07-b7f6-30577dfeec8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:44.733948 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-b663a430-1842-4f07-b7f6-30577dfeec8f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:44.735168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b663a430-1842-4f07-b7f6-30577dfeec8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:44.798566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1352/2717] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:44.806634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cb4bc-e3ed-4b59-89b6-ebfdcc0ce56f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:44.806920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cb4bc-e3ed-4b59-89b6-ebfdcc0ce56f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:44.807202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cb4bc-e3ed-4b59-89b6-ebfdcc0ce56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:44.832191 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-a33cb4bc-e3ed-4b59-89b6-ebfdcc0ce56f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:44.833473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-a33cb4bc-e3ed-4b59-89b6-ebfdcc0ce56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:44.842816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1366/2718] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:44.848438 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663a430-1842-4f07-b7f6-30577dfeec8f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:44.858597 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663a430-1842-4f07-b7f6-30577dfeec8f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:44.863448 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f0018-572d-4d07-8e65-e66d195b0ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:44.864152 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f0018-572d-4d07-8e65-e66d195b0ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:44.864429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f0018-572d-4d07-8e65-e66d195b0ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:44.864738 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f0018-572d-4d07-8e65-e66d195b0ed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:44.865471 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f0018-572d-4d07-8e65-e66d195b0ed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:58:44.865809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f0018-572d-4d07-8e65-e66d195b0ed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:44.867401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f0018-572d-4d07-8e65-e66d195b0ed0 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:44.880368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1353/2719] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:58:44.884787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cb4bc-e3ed-4b59-89b6-ebfdcc0ce56f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:44.895100 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cb4bc-e3ed-4b59-89b6-ebfdcc0ce56f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:44.899875 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36313e29-0104-43cc-9c27-4c597294858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:44.900536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36313e29-0104-43cc-9c27-4c597294858c admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:44.900856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36313e29-0104-43cc-9c27-4c597294858c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:44.901136 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36313e29-0104-43cc-9c27-4c597294858c admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:44.901832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36313e29-0104-43cc-9c27-4c597294858c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:44.903724 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36313e29-0104-43cc-9c27-4c597294858c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:44.935620 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1367/2720] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:58:44 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:58:44.936808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f0018-572d-4d07-8e65-e66d195b0ed0 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:44.947802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f0018-572d-4d07-8e65-e66d195b0ed0 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:45.008216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6915a75-0ad3-4cc4-8508-3bbc87b38e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:45.008841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6915a75-0ad3-4cc4-8508-3bbc87b38e95 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.009130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6915a75-0ad3-4cc4-8508-3bbc87b38e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.009408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6915a75-0ad3-4cc4-8508-3bbc87b38e95 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.010128 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6915a75-0ad3-4cc4-8508-3bbc87b38e95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:45.013967 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6915a75-0ad3-4cc4-8508-3bbc87b38e95 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:45.035234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1354/2721] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:58:44 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:58:45.039669 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36313e29-0104-43cc-9c27-4c597294858c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:45.050892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36313e29-0104-43cc-9c27-4c597294858c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:45.057839 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2caf3-1fca-4cd1-8ec4-58b613cac55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:45.058515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2caf3-1fca-4cd1-8ec4-58b613cac55c admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.058801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2caf3-1fca-4cd1-8ec4-58b613cac55c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.059081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2caf3-1fca-4cd1-8ec4-58b613cac55c admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.059816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2caf3-1fca-4cd1-8ec4-58b613cac55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:45.061460 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2caf3-1fca-4cd1-8ec4-58b613cac55c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:45.073537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1368/2722] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:45.077877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6915a75-0ad3-4cc4-8508-3bbc87b38e95 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:45.088112 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6915a75-0ad3-4cc4-8508-3bbc87b38e95 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:45.093515 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8052a9-ed7f-47ff-99fe-28ae8f66b4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:45.094212 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8052a9-ed7f-47ff-99fe-28ae8f66b4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.094503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8052a9-ed7f-47ff-99fe-28ae8f66b4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.094843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8052a9-ed7f-47ff-99fe-28ae8f66b4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.095533 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8052a9-ed7f-47ff-99fe-28ae8f66b4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:45.097221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8052a9-ed7f-47ff-99fe-28ae8f66b4c1 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:45.099781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:45.114857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1355/2723] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:45.115880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2caf3-1fca-4cd1-8ec4-58b613cac55c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:45.127309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2caf3-1fca-4cd1-8ec4-58b613cac55c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:45.132150 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e28a6-696e-4932-9e1d-078553652763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:45.133090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e28a6-696e-4932-9e1d-078553652763 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.133382 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e28a6-696e-4932-9e1d-078553652763 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.133659 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e28a6-696e-4932-9e1d-078553652763 admin admin] PATH_INFO: `/v3/projects/2f092125a7464a3fa38bb3f1210bc878/users/667348aad397446dbe57c26134aa8f09/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.149704 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e28a6-696e-4932-9e1d-078553652763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f092125a7464a3fa38bb3f1210bc878, user_id=667348aad397446dbe57c26134aa8f09, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:45.152401 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e28a6-696e-4932-9e1d-078553652763 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:45.168205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1369/2724] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:58:45 2022] PUT /identity/v3/projects/2f092125a7464a3fa38bb3f1210bc878/users/667348aad397446dbe57c26134aa8f09/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:58:45.179799 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfc1a1-0b1c-45cc-bbc9-5f6a6239c97d None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.180076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfc1a1-0b1c-45cc-bbc9-5f6a6239c97d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.180354 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfc1a1-0b1c-45cc-bbc9-5f6a6239c97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.213664 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-c4dfc1a1-0b1c-45cc-bbc9-5f6a6239c97d None None] MFA Rules not processed for user `667348aad397446dbe57c26134aa8f09`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:45.215042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c4dfc1a1-0b1c-45cc-bbc9-5f6a6239c97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:45.285355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1356/2725] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:45.290048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e28a6-696e-4932-9e1d-078553652763 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:45.359638 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e28a6-696e-4932-9e1d-078553652763 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:45.364430 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f32f1-c47d-4d37-9f3b-ff2f975d2eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:45.365128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f32f1-c47d-4d37-9f3b-ff2f975d2eab admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.365412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f32f1-c47d-4d37-9f3b-ff2f975d2eab admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.365688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f32f1-c47d-4d37-9f3b-ff2f975d2eab admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.366398 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f32f1-c47d-4d37-9f3b-ff2f975d2eab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:45.368307 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f32f1-c47d-4d37-9f3b-ff2f975d2eab admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:45.402234 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1370/2726] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:58:45 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:58:45.406126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfc1a1-0b1c-45cc-bbc9-5f6a6239c97d None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:45.418042 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfc1a1-0b1c-45cc-bbc9-5f6a6239c97d None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:45.481940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396aef24-7d59-48d8-a700-676eeb47b57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:45.482992 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396aef24-7d59-48d8-a700-676eeb47b57c admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.483462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396aef24-7d59-48d8-a700-676eeb47b57c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.483909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396aef24-7d59-48d8-a700-676eeb47b57c admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.485189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396aef24-7d59-48d8-a700-676eeb47b57c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:45.488349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396aef24-7d59-48d8-a700-676eeb47b57c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:45.511683 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1357/2727] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:58:45 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:58:45.516456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f32f1-c47d-4d37-9f3b-ff2f975d2eab admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:45.527977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f32f1-c47d-4d37-9f3b-ff2f975d2eab admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:45.532801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f486b6-1260-4a09-9d40-7661f43da1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:45.533467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f486b6-1260-4a09-9d40-7661f43da1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.533754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f486b6-1260-4a09-9d40-7661f43da1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.534033 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f486b6-1260-4a09-9d40-7661f43da1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.534768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f486b6-1260-4a09-9d40-7661f43da1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:45.536451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f486b6-1260-4a09-9d40-7661f43da1d1 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:45.537860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:45.552763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1371/2728] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:45.553985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396aef24-7d59-48d8-a700-676eeb47b57c admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:45.565392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396aef24-7d59-48d8-a700-676eeb47b57c admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:45.571137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e51807-89a0-4800-b9c6-d5136b964211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:45.571825 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e51807-89a0-4800-b9c6-d5136b964211 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.572110 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e51807-89a0-4800-b9c6-d5136b964211 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.572388 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e51807-89a0-4800-b9c6-d5136b964211 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.573170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e51807-89a0-4800-b9c6-d5136b964211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:45.574788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e51807-89a0-4800-b9c6-d5136b964211 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:45.586541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1358/2729] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:58:45.591002 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f486b6-1260-4a09-9d40-7661f43da1d1 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:45.601778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f486b6-1260-4a09-9d40-7661f43da1d1 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:45.606640 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a515f3-516c-4851-967d-269ee4f9d535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:45.607534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a515f3-516c-4851-967d-269ee4f9d535 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.607813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a515f3-516c-4851-967d-269ee4f9d535 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.608092 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a515f3-516c-4851-967d-269ee4f9d535 admin admin] PATH_INFO: `/v3/projects/0f7a9c51aa8248fc9b69363024f9cff3/users/fafc6f32f31541f0b0add7998ac1b342/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.624855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a515f3-516c-4851-967d-269ee4f9d535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f7a9c51aa8248fc9b69363024f9cff3, user_id=fafc6f32f31541f0b0add7998ac1b342, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:45.628061 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a515f3-516c-4851-967d-269ee4f9d535 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:45.649921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1372/2730] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:58:45 2022] PUT /identity/v3/projects/0f7a9c51aa8248fc9b69363024f9cff3/users/fafc6f32f31541f0b0add7998ac1b342/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:45.663202 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be64e7-5972-4b44-8e9a-9a787969a580 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.663494 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be64e7-5972-4b44-8e9a-9a787969a580 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.663776 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be64e7-5972-4b44-8e9a-9a787969a580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.698249 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-25be64e7-5972-4b44-8e9a-9a787969a580 None None] MFA Rules not processed for user `fafc6f32f31541f0b0add7998ac1b342`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:45.699508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-25be64e7-5972-4b44-8e9a-9a787969a580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:45.762635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:45.778485 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1359/2731] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:45.782653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a515f3-516c-4851-967d-269ee4f9d535 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:45.849238 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a515f3-516c-4851-967d-269ee4f9d535 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:45.853999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097b918-d430-4ee7-bfcb-46d63809b6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:45.854667 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097b918-d430-4ee7-bfcb-46d63809b6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.854954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097b918-d430-4ee7-bfcb-46d63809b6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.855236 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097b918-d430-4ee7-bfcb-46d63809b6e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.855944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097b918-d430-4ee7-bfcb-46d63809b6e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:45.857926 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097b918-d430-4ee7-bfcb-46d63809b6e5 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:45.890435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1373/2732] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:58:45 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:58:45.891690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be64e7-5972-4b44-8e9a-9a787969a580 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:45.903837 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be64e7-5972-4b44-8e9a-9a787969a580 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:45.970928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13223ad0-2553-4a95-97f2-b8e49b1166f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:45.971662 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13223ad0-2553-4a95-97f2-b8e49b1166f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:45.971944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13223ad0-2553-4a95-97f2-b8e49b1166f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:45.972217 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13223ad0-2553-4a95-97f2-b8e49b1166f0 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:45.973138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13223ad0-2553-4a95-97f2-b8e49b1166f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:45.975101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13223ad0-2553-4a95-97f2-b8e49b1166f0 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:45.998044 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1360/2733] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:58:45 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:58:46.002688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097b918-d430-4ee7-bfcb-46d63809b6e5 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:46.013537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097b918-d430-4ee7-bfcb-46d63809b6e5 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:46.018394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6260591-e3f4-44fd-b85d-6b374eed4e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:46.019073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6260591-e3f4-44fd-b85d-6b374eed4e8b admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:46.019354 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6260591-e3f4-44fd-b85d-6b374eed4e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:46.019629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6260591-e3f4-44fd-b85d-6b374eed4e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:46.020364 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6260591-e3f4-44fd-b85d-6b374eed4e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:46.022013 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6260591-e3f4-44fd-b85d-6b374eed4e8b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:46.033378 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1374/2734] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:58:46.037793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13223ad0-2553-4a95-97f2-b8e49b1166f0 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:46.048692 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13223ad0-2553-4a95-97f2-b8e49b1166f0 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:46.053479 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c669aca-3fed-4224-90b5-55eb3ea901b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:46.054380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c669aca-3fed-4224-90b5-55eb3ea901b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:46.054674 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c669aca-3fed-4224-90b5-55eb3ea901b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:46.054954 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c669aca-3fed-4224-90b5-55eb3ea901b8 admin admin] PATH_INFO: `/v3/projects/914d6c4183a44eb5b05bfa9a183e2a41/users/c1240cf485f04861878e93002df940e2/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:46.074369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c669aca-3fed-4224-90b5-55eb3ea901b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=914d6c4183a44eb5b05bfa9a183e2a41, user_id=c1240cf485f04861878e93002df940e2, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:46.077065 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c669aca-3fed-4224-90b5-55eb3ea901b8 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:46.091092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1361/2735] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:58:46 2022] PUT /identity/v3/projects/914d6c4183a44eb5b05bfa9a183e2a41/users/c1240cf485f04861878e93002df940e2/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:58:46.095827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6260591-e3f4-44fd-b85d-6b374eed4e8b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:46.106503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6260591-e3f4-44fd-b85d-6b374eed4e8b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:46.177381 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d3cae-5db3-4a3e-9c9d-5def3fc287b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:46.178099 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d3cae-5db3-4a3e-9c9d-5def3fc287b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:46.178332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d3cae-5db3-4a3e-9c9d-5def3fc287b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:46.178613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d3cae-5db3-4a3e-9c9d-5def3fc287b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:46.179349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d3cae-5db3-4a3e-9c9d-5def3fc287b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:46.181001 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d3cae-5db3-4a3e-9c9d-5def3fc287b8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:46.196376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1375/2736] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:58:46.197535 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c669aca-3fed-4224-90b5-55eb3ea901b8 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:46.209008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c669aca-3fed-4224-90b5-55eb3ea901b8 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:46.213732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560158dd-3700-42ad-8c80-27037236727f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:46.214633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560158dd-3700-42ad-8c80-27037236727f admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:46.214921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560158dd-3700-42ad-8c80-27037236727f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:46.215196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560158dd-3700-42ad-8c80-27037236727f admin admin] PATH_INFO: `/v3/projects/914d6c4183a44eb5b05bfa9a183e2a41/users/c1240cf485f04861878e93002df940e2/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:46.217283 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560158dd-3700-42ad-8c80-27037236727f admin admin] RBAC: Authorizing `identity:create_grant(project_id=914d6c4183a44eb5b05bfa9a183e2a41, user_id=c1240cf485f04861878e93002df940e2, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:46.219961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560158dd-3700-42ad-8c80-27037236727f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:46.230758 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1362/2737] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:58:46 2022] PUT /identity/v3/projects/914d6c4183a44eb5b05bfa9a183e2a41/users/c1240cf485f04861878e93002df940e2/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:58:46.241382 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bcf36-acab-4d4d-a861-5836f94241d3 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:46.241672 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bcf36-acab-4d4d-a861-5836f94241d3 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:46.241954 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bcf36-acab-4d4d-a861-5836f94241d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:46.272269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-329bcf36-acab-4d4d-a861-5836f94241d3 None None] MFA Rules not processed for user `c1240cf485f04861878e93002df940e2`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:58:46.273553 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-329bcf36-acab-4d4d-a861-5836f94241d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:46.311208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:46.357050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1376/2738] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:58:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:58:46.368188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560158dd-3700-42ad-8c80-27037236727f admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:46.370262 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-560158dd-3700-42ad-8c80-27037236727f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:46.431442 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560158dd-3700-42ad-8c80-27037236727f admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:46.436235 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac96689-28d0-4080-ab5a-bf4f03a0f9e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:46.436956 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac96689-28d0-4080-ab5a-bf4f03a0f9e4 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:46.437245 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac96689-28d0-4080-ab5a-bf4f03a0f9e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:46.437574 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac96689-28d0-4080-ab5a-bf4f03a0f9e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:46.448667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac96689-28d0-4080-ab5a-bf4f03a0f9e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:46.450736 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac96689-28d0-4080-ab5a-bf4f03a0f9e4 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:46.461343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1363/2739] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:58:47.019276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bcf36-acab-4d4d-a861-5836f94241d3 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:47.020755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-329bcf36-acab-4d4d-a861-5836f94241d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:58:47.033629 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bcf36-acab-4d4d-a861-5836f94241d3 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:47.037339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b9505-c669-4e96-9dc6-9900d986a35d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:47.037999 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b9505-c669-4e96-9dc6-9900d986a35d service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:47.038285 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b9505-c669-4e96-9dc6-9900d986a35d service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:47.038562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b9505-c669-4e96-9dc6-9900d986a35d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:47.099808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b9505-c669-4e96-9dc6-9900d986a35d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:47.101359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b9505-c669-4e96-9dc6-9900d986a35d service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:47.113298 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1377/2740] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:58:52.236841 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac96689-28d0-4080-ab5a-bf4f03a0f9e4 service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:58:52.246230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac96689-28d0-4080-ab5a-bf4f03a0f9e4 service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:58:52.250161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb13e0d-f127-44f5-9a71-6b19e634a5e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:58:52.250815 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb13e0d-f127-44f5-9a71-6b19e634a5e9 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:58:52.251101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb13e0d-f127-44f5-9a71-6b19e634a5e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:58:52.251379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb13e0d-f127-44f5-9a71-6b19e634a5e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:58:52.296834 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:58:52.316383 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb13e0d-f127-44f5-9a71-6b19e634a5e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:58:52.317957 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb13e0d-f127-44f5-9a71-6b19e634a5e9 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:58:52.328162 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1364/2741] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:58:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:58:57.106753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=43 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:00.605995 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b9505-c669-4e96-9dc6-9900d986a35d service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:00.685679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b9505-c669-4e96-9dc6-9900d986a35d service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:00.690758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:00.691631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:00.691920 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:00.692198 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] PATH_INFO: `/v3/users/667348aad397446dbe57c26134aa8f09` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:00.693444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=667348aad397446dbe57c26134aa8f09)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:00.695384 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:00.747421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '667348aad397446dbe57c26134aa8f09'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:00.748161 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '667348aad397446dbe57c26134aa8f09'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:00.770494 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '667348aad397446dbe57c26134aa8f09'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:00.792521 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '667348aad397446dbe57c26134aa8f09'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:00.805058 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '667348aad397446dbe57c26134aa8f09'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:00.811190 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1378/2742] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:59:00 2022] DELETE /identity/v3/users/667348aad397446dbe57c26134aa8f09 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:00.816307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497022f1-11c2-4a07-91f6-f76ed4b59418 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:00.816830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497022f1-11c2-4a07-91f6-f76ed4b59418 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:00.816950 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497022f1-11c2-4a07-91f6-f76ed4b59418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:00.836808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-497022f1-11c2-4a07-91f6-f76ed4b59418 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:00.838030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-497022f1-11c2-4a07-91f6-f76ed4b59418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:00.899570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1365/2743] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:59:00.909529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:00.910953 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:00.976347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66760b10-31d0-442a-8d53-45458b374fd8 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:00.980063 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e657fcb-15ec-4033-9010-126a7e95fddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:00.980759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e657fcb-15ec-4033-9010-126a7e95fddb service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:00.981043 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e657fcb-15ec-4033-9010-126a7e95fddb service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:00.981316 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e657fcb-15ec-4033-9010-126a7e95fddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:00.991356 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e657fcb-15ec-4033-9010-126a7e95fddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:00.992823 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e657fcb-15ec-4033-9010-126a7e95fddb service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:01.003324 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1379/2744] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:01.287464 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497022f1-11c2-4a07-91f6-f76ed4b59418 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:01.289223 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-497022f1-11c2-4a07-91f6-f76ed4b59418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:01.309294 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497022f1-11c2-4a07-91f6-f76ed4b59418 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:01.314712 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:01.315557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:01.315843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:01.316167 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] PATH_INFO: `/v3/projects/2f092125a7464a3fa38bb3f1210bc878` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:01.317372 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f092125a7464a3fa38bb3f1210bc878)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:01.319298 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:01.374066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f092125a7464a3fa38bb3f1210bc878', 'request_id': 'req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:01.382319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f092125a7464a3fa38bb3f1210bc878', 'request_id': 'req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:01.387891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f092125a7464a3fa38bb3f1210bc878', 'request_id': 'req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:01.400192 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] The token cache is being invalidate because project 2f092125a7464a3fa38bb3f1210bc878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:59:01.400509 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:01.401200 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:01.404130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1366/2745] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:59:01 2022] DELETE /identity/v3/projects/2f092125a7464a3fa38bb3f1210bc878 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:01.406979 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e657fcb-15ec-4033-9010-126a7e95fddb service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:01.416992 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e657fcb-15ec-4033-9010-126a7e95fddb service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:01.436893 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:01.478883 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:01.479861 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:01.480168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:01.480430 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] PATH_INFO: `/v3/users/fafc6f32f31541f0b0add7998ac1b342` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:01.481730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fafc6f32f31541f0b0add7998ac1b342)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:01.483685 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:01.532977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fafc6f32f31541f0b0add7998ac1b342'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:01.533647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fafc6f32f31541f0b0add7998ac1b342'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:01.540788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fafc6f32f31541f0b0add7998ac1b342'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:01.555306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fafc6f32f31541f0b0add7998ac1b342'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:01.562435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fafc6f32f31541f0b0add7998ac1b342'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:01.569257 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1380/2746] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:59:01 2022] DELETE /identity/v3/users/fafc6f32f31541f0b0add7998ac1b342 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:01.857806 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:01.859493 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:01.876363 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c307a15-3c9d-4c6f-86e3-a9ee5d8a03b7 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:01.955793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:01.956715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:01.956997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:01.957269 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] PATH_INFO: `/v3/projects/0f7a9c51aa8248fc9b69363024f9cff3` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:01.958498 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f7a9c51aa8248fc9b69363024f9cff3)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:01.960456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:01.972905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:02.018892 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f7a9c51aa8248fc9b69363024f9cff3', 'request_id': 'req-75e3c4cd-bd9c-497c-b448-7663b47f867e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.028801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f7a9c51aa8248fc9b69363024f9cff3', 'request_id': 'req-75e3c4cd-bd9c-497c-b448-7663b47f867e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.037054 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f7a9c51aa8248fc9b69363024f9cff3', 'request_id': 'req-75e3c4cd-bd9c-497c-b448-7663b47f867e'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.049909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] The token cache is being invalidate because project 0f7a9c51aa8248fc9b69363024f9cff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:59:02.050205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.050893 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.052724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1367/2747] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:59:01 2022] DELETE /identity/v3/projects/0f7a9c51aa8248fc9b69363024f9cff3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:02.058772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:02.061862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:02.086796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab3431-7140-4917-9e93-bdf9ec7ae686 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:02.175567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f66e0a-15b6-47dd-b07e-bcc02ab79d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:02.178328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f66e0a-15b6-47dd-b07e-bcc02ab79d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:02.178328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f66e0a-15b6-47dd-b07e-bcc02ab79d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:02.178328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f66e0a-15b6-47dd-b07e-bcc02ab79d53 admin admin] PATH_INFO: `/v3/users/c1240cf485f04861878e93002df940e2` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:02.179799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f66e0a-15b6-47dd-b07e-bcc02ab79d53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1240cf485f04861878e93002df940e2)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:02.181923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f66e0a-15b6-47dd-b07e-bcc02ab79d53 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:02.249140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:02.251803 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-79f66e0a-15b6-47dd-b07e-bcc02ab79d53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1240cf485f04861878e93002df940e2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.254628 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-79f66e0a-15b6-47dd-b07e-bcc02ab79d53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1240cf485f04861878e93002df940e2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.264363 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-79f66e0a-15b6-47dd-b07e-bcc02ab79d53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1240cf485f04861878e93002df940e2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.278171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-79f66e0a-15b6-47dd-b07e-bcc02ab79d53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1240cf485f04861878e93002df940e2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.285828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-79f66e0a-15b6-47dd-b07e-bcc02ab79d53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1240cf485f04861878e93002df940e2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.292346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1381/2748] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:59:02 2022] DELETE /identity/v3/users/c1240cf485f04861878e93002df940e2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:02.577197 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:02.578700 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:02.593340 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3c4cd-bd9c-497c-b448-7663b47f867e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:02.657564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:02.658447 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:02.658768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:02.659032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362 admin admin] PATH_INFO: `/v3/projects/914d6c4183a44eb5b05bfa9a183e2a41` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:02.660248 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=914d6c4183a44eb5b05bfa9a183e2a41)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:02.662172 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:02.723018 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '914d6c4183a44eb5b05bfa9a183e2a41', 'request_id': 'req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.732136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '914d6c4183a44eb5b05bfa9a183e2a41', 'request_id': 'req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.738070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '914d6c4183a44eb5b05bfa9a183e2a41', 'request_id': 'req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.750041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362 admin admin] The token cache is being invalidate because project 914d6c4183a44eb5b05bfa9a183e2a41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:59:02.750631 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.751292 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0a1c4b7d-b9e0-4e52-ac70-774f293c6362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:02.753382 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1368/2749] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:59:02 2022] DELETE /identity/v3/projects/914d6c4183a44eb5b05bfa9a183e2a41 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:02.759336 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e51b2-418f-49d3-9957-5687646f1001 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:02.759658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e51b2-418f-49d3-9957-5687646f1001 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:02.760066 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e51b2-418f-49d3-9957-5687646f1001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:02.780328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-fa2e51b2-418f-49d3-9957-5687646f1001 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:02.781592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fa2e51b2-418f-49d3-9957-5687646f1001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:02.850258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1382/2750] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:02.858871 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11e6c3-6e06-4982-84de-6b51d57e2228 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:02.859428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11e6c3-6e06-4982-84de-6b51d57e2228 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:02.859428 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11e6c3-6e06-4982-84de-6b51d57e2228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:02.880755 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-7b11e6c3-6e06-4982-84de-6b51d57e2228 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:02.881997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7b11e6c3-6e06-4982-84de-6b51d57e2228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:02.892288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1369/2751] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:02.897199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e51b2-418f-49d3-9957-5687646f1001 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:02.907219 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e51b2-418f-49d3-9957-5687646f1001 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:02.911915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cbbf7-2ab0-43d0-9142-9772568d64b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:02.912591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cbbf7-2ab0-43d0-9142-9772568d64b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:02.912897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cbbf7-2ab0-43d0-9142-9772568d64b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:02.913169 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cbbf7-2ab0-43d0-9142-9772568d64b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:02.913936 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cbbf7-2ab0-43d0-9142-9772568d64b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:59:02.914244 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cbbf7-2ab0-43d0-9142-9772568d64b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:02.915844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cbbf7-2ab0-43d0-9142-9772568d64b6 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:02.928097 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1383/2752] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:59:02.932691 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11e6c3-6e06-4982-84de-6b51d57e2228 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:02.942743 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11e6c3-6e06-4982-84de-6b51d57e2228 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:02.947585 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41b85b-590b-4132-8ea1-ca735e90f515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:02.948243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41b85b-590b-4132-8ea1-ca735e90f515 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:02.948521 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41b85b-590b-4132-8ea1-ca735e90f515 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:02.948936 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41b85b-590b-4132-8ea1-ca735e90f515 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:02.949641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41b85b-590b-4132-8ea1-ca735e90f515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:02.951561 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41b85b-590b-4132-8ea1-ca735e90f515 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:02.988043 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1370/2753] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:59:02 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:02.992237 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cbbf7-2ab0-43d0-9142-9772568d64b6 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:03.003820 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cbbf7-2ab0-43d0-9142-9772568d64b6 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:03.066778 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156db1fc-d219-4c2d-8f97-8673a775ae28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:03.067486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156db1fc-d219-4c2d-8f97-8673a775ae28 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:03.067729 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156db1fc-d219-4c2d-8f97-8673a775ae28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:03.068006 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156db1fc-d219-4c2d-8f97-8673a775ae28 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:03.068744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156db1fc-d219-4c2d-8f97-8673a775ae28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:03.070698 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156db1fc-d219-4c2d-8f97-8673a775ae28 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:03.092517 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1384/2754] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:59:02 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:03.097870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41b85b-590b-4132-8ea1-ca735e90f515 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:03.103571 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:03.110213 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41b85b-590b-4132-8ea1-ca735e90f515 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:03.117928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdeeed-796e-4926-adae-66b4087fb998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:03.118616 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdeeed-796e-4926-adae-66b4087fb998 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:03.118907 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdeeed-796e-4926-adae-66b4087fb998 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:03.119191 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdeeed-796e-4926-adae-66b4087fb998 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:03.119973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdeeed-796e-4926-adae-66b4087fb998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:03.121885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdeeed-796e-4926-adae-66b4087fb998 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:03.133717 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1371/2755] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:03.138166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156db1fc-d219-4c2d-8f97-8673a775ae28 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:03.148486 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156db1fc-d219-4c2d-8f97-8673a775ae28 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:03.153306 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb284cb7-5843-4208-b0d4-604859f56991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:03.153973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb284cb7-5843-4208-b0d4-604859f56991 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:03.154255 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb284cb7-5843-4208-b0d4-604859f56991 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:03.154532 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb284cb7-5843-4208-b0d4-604859f56991 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:03.155358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb284cb7-5843-4208-b0d4-604859f56991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:03.156995 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb284cb7-5843-4208-b0d4-604859f56991 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:03.167439 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1385/2756] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:59:03.171772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdeeed-796e-4926-adae-66b4087fb998 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:03.182588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdeeed-796e-4926-adae-66b4087fb998 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:03.188199 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b41c59-3458-4e1d-8be6-268083cbf233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:03.189230 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b41c59-3458-4e1d-8be6-268083cbf233 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:03.189511 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b41c59-3458-4e1d-8be6-268083cbf233 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:03.190392 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b41c59-3458-4e1d-8be6-268083cbf233 admin admin] PATH_INFO: `/v3/projects/a62ba42b831b4cf39a90f8aa9e24cb16/users/490fdd35c38f448f845c578dcb0f37bb/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:03.205981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b41c59-3458-4e1d-8be6-268083cbf233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62ba42b831b4cf39a90f8aa9e24cb16, user_id=490fdd35c38f448f845c578dcb0f37bb, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:03.208684 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b41c59-3458-4e1d-8be6-268083cbf233 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:03.224151 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1372/2757] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:03 2022] PUT /identity/v3/projects/a62ba42b831b4cf39a90f8aa9e24cb16/users/490fdd35c38f448f845c578dcb0f37bb/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:03.237585 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60935e-3bff-4f9c-9713-51cb230f76d5 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:03.237871 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60935e-3bff-4f9c-9713-51cb230f76d5 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:03.238271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60935e-3bff-4f9c-9713-51cb230f76d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:03.269497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-4b60935e-3bff-4f9c-9713-51cb230f76d5 None None] MFA Rules not processed for user `490fdd35c38f448f845c578dcb0f37bb`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:03.270880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4b60935e-3bff-4f9c-9713-51cb230f76d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:03.286816 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:03.345069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1386/2758] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:03.356799 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b41c59-3458-4e1d-8be6-268083cbf233 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:03.359096 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-74b41c59-3458-4e1d-8be6-268083cbf233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:03.426174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b41c59-3458-4e1d-8be6-268083cbf233 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:03.430324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d05b14-8f96-4480-833e-cbdc3e5382e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:03.430993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d05b14-8f96-4480-833e-cbdc3e5382e1 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:03.431278 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d05b14-8f96-4480-833e-cbdc3e5382e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:03.431557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d05b14-8f96-4480-833e-cbdc3e5382e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:03.441205 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d05b14-8f96-4480-833e-cbdc3e5382e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:03.442707 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d05b14-8f96-4480-833e-cbdc3e5382e1 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:03.452588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1373/2759] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:03.492833 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60935e-3bff-4f9c-9713-51cb230f76d5 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:03.582844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60935e-3bff-4f9c-9713-51cb230f76d5 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:03.587829 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:03.589377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:03.589899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:03.590187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] PATH_INFO: `/v3/users/490fdd35c38f448f845c578dcb0f37bb` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:03.592343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=490fdd35c38f448f845c578dcb0f37bb)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:03.595798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:03.650794 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '490fdd35c38f448f845c578dcb0f37bb'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:03.651494 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '490fdd35c38f448f845c578dcb0f37bb'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:03.660928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '490fdd35c38f448f845c578dcb0f37bb'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:03.675069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '490fdd35c38f448f845c578dcb0f37bb'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:03.681951 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '490fdd35c38f448f845c578dcb0f37bb'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:03.688164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1387/2760] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:59:03 2022] DELETE /identity/v3/users/490fdd35c38f448f845c578dcb0f37bb => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:03.692823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c995-ecbe-4b89-a882-39ca88d6167e None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:03.693117 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c995-ecbe-4b89-a882-39ca88d6167e None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:03.693403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c995-ecbe-4b89-a882-39ca88d6167e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:03.712637 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-d421c995-ecbe-4b89-a882-39ca88d6167e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:03.713864 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d421c995-ecbe-4b89-a882-39ca88d6167e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:03.774002 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1374/2761] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:03.784197 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:03.785690 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:03.825610 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:03.847913 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4853ad7-e6b3-455f-a252-4dfd2d643099 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:03.852020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e12587-62df-455c-a132-c9b35db5f6a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:03.852768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e12587-62df-455c-a132-c9b35db5f6a0 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:03.853054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e12587-62df-455c-a132-c9b35db5f6a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:03.853359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e12587-62df-455c-a132-c9b35db5f6a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:03.863399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e12587-62df-455c-a132-c9b35db5f6a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:03.864932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e12587-62df-455c-a132-c9b35db5f6a0 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:03.875776 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1388/2762] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:04.150010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c995-ecbe-4b89-a882-39ca88d6167e None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:04.153274 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-d421c995-ecbe-4b89-a882-39ca88d6167e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:04.166441 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c995-ecbe-4b89-a882-39ca88d6167e None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:04.171336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323f94f-468e-470d-a932-4599bac5c1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:04.172252 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323f94f-468e-470d-a932-4599bac5c1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:04.172531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323f94f-468e-470d-a932-4599bac5c1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:04.172836 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323f94f-468e-470d-a932-4599bac5c1d3 admin admin] PATH_INFO: `/v3/projects/a62ba42b831b4cf39a90f8aa9e24cb16` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:04.174067 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323f94f-468e-470d-a932-4599bac5c1d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a62ba42b831b4cf39a90f8aa9e24cb16)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:04.176070 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323f94f-468e-470d-a932-4599bac5c1d3 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:04.227397 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9323f94f-468e-470d-a932-4599bac5c1d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a62ba42b831b4cf39a90f8aa9e24cb16', 'request_id': 'req-9323f94f-468e-470d-a932-4599bac5c1d3'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:04.236349 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9323f94f-468e-470d-a932-4599bac5c1d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a62ba42b831b4cf39a90f8aa9e24cb16', 'request_id': 'req-9323f94f-468e-470d-a932-4599bac5c1d3'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:04.242255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9323f94f-468e-470d-a932-4599bac5c1d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a62ba42b831b4cf39a90f8aa9e24cb16', 'request_id': 'req-9323f94f-468e-470d-a932-4599bac5c1d3'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:04.243348 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:04.258164 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9323f94f-468e-470d-a932-4599bac5c1d3 admin admin] The token cache is being invalidate because project a62ba42b831b4cf39a90f8aa9e24cb16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:59:04.258452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9323f94f-468e-470d-a932-4599bac5c1d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:04.259116 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-9323f94f-468e-470d-a932-4599bac5c1d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:04.260718 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1375/2763] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:59:04 2022] DELETE /identity/v3/projects/a62ba42b831b4cf39a90f8aa9e24cb16 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:04.267222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855250e-0121-434c-8c4d-a93d9387a8cc None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:04.267734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855250e-0121-434c-8c4d-a93d9387a8cc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:04.267914 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855250e-0121-434c-8c4d-a93d9387a8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:04.290401 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d855250e-0121-434c-8c4d-a93d9387a8cc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:04.291534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d855250e-0121-434c-8c4d-a93d9387a8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:04.352777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1389/2764] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:59:04.360803 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed01113-2321-4c98-8613-25e170c30560 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:04.361101 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed01113-2321-4c98-8613-25e170c30560 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:04.361433 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed01113-2321-4c98-8613-25e170c30560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:04.383546 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-6ed01113-2321-4c98-8613-25e170c30560 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:04.384828 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-6ed01113-2321-4c98-8613-25e170c30560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:04.393617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1376/2765] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:59:04.398222 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855250e-0121-434c-8c4d-a93d9387a8cc None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:04.408390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855250e-0121-434c-8c4d-a93d9387a8cc None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:04.413189 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383050c-bfbe-4955-8099-9a073293bb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:04.413859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383050c-bfbe-4955-8099-9a073293bb4b admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:04.414151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383050c-bfbe-4955-8099-9a073293bb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:04.414436 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383050c-bfbe-4955-8099-9a073293bb4b admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:04.415171 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383050c-bfbe-4955-8099-9a073293bb4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:59:04.415481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383050c-bfbe-4955-8099-9a073293bb4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:04.417143 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383050c-bfbe-4955-8099-9a073293bb4b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:04.429962 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1390/2766] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:59:04.434445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed01113-2321-4c98-8613-25e170c30560 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:04.444702 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed01113-2321-4c98-8613-25e170c30560 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:04.449445 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdcc1fa-4d03-4caa-b477-0d83b6729227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:04.450115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdcc1fa-4d03-4caa-b477-0d83b6729227 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:04.450405 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdcc1fa-4d03-4caa-b477-0d83b6729227 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:04.450697 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdcc1fa-4d03-4caa-b477-0d83b6729227 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:04.451398 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdcc1fa-4d03-4caa-b477-0d83b6729227 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:04.453346 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdcc1fa-4d03-4caa-b477-0d83b6729227 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:04.482737 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1377/2767] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:59:04 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:04.487508 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383050c-bfbe-4955-8099-9a073293bb4b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:04.498242 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383050c-bfbe-4955-8099-9a073293bb4b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:04.562004 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c9247-fc88-4f66-89c3-90c8970f3b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:04.562594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c9247-fc88-4f66-89c3-90c8970f3b5b admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:04.562879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c9247-fc88-4f66-89c3-90c8970f3b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:04.563159 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c9247-fc88-4f66-89c3-90c8970f3b5b admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:04.563874 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c9247-fc88-4f66-89c3-90c8970f3b5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:04.566473 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c9247-fc88-4f66-89c3-90c8970f3b5b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:04.587149 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1391/2768] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:59:04 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:59:04.591334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdcc1fa-4d03-4caa-b477-0d83b6729227 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:04.602126 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdcc1fa-4d03-4caa-b477-0d83b6729227 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:04.606870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e13ae3-9af0-4fd4-b09a-c7d50770ecda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:04.607539 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e13ae3-9af0-4fd4-b09a-c7d50770ecda admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:04.607823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e13ae3-9af0-4fd4-b09a-c7d50770ecda admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:04.608099 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e13ae3-9af0-4fd4-b09a-c7d50770ecda admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:04.608875 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e13ae3-9af0-4fd4-b09a-c7d50770ecda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:04.610490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e13ae3-9af0-4fd4-b09a-c7d50770ecda admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:04.626852 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1378/2769] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:04.628738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c9247-fc88-4f66-89c3-90c8970f3b5b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:04.639787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c9247-fc88-4f66-89c3-90c8970f3b5b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:04.644603 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a352c55-621b-4908-a3f8-5a6d9319836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:04.645266 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a352c55-621b-4908-a3f8-5a6d9319836b admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:04.645552 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a352c55-621b-4908-a3f8-5a6d9319836b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:04.646259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a352c55-621b-4908-a3f8-5a6d9319836b admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:04.646556 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a352c55-621b-4908-a3f8-5a6d9319836b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:04.648151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a352c55-621b-4908-a3f8-5a6d9319836b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:04.659907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1392/2770] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:04.664745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e13ae3-9af0-4fd4-b09a-c7d50770ecda admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:04.675735 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e13ae3-9af0-4fd4-b09a-c7d50770ecda admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:04.682633 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cb958-6074-48d0-b0ef-fffde9f06ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:04.683524 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cb958-6074-48d0-b0ef-fffde9f06ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:04.683807 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cb958-6074-48d0-b0ef-fffde9f06ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:04.684086 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cb958-6074-48d0-b0ef-fffde9f06ba2 admin admin] PATH_INFO: `/v3/projects/4a77723e8867459c99fcd86914fa44df/users/52e9d0e3c2dd453eb4bfc2db6b45d80f/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:04.708079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cb958-6074-48d0-b0ef-fffde9f06ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a77723e8867459c99fcd86914fa44df, user_id=52e9d0e3c2dd453eb4bfc2db6b45d80f, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:04.710781 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cb958-6074-48d0-b0ef-fffde9f06ba2 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:04.725446 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1379/2771] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:04 2022] PUT /identity/v3/projects/4a77723e8867459c99fcd86914fa44df/users/52e9d0e3c2dd453eb4bfc2db6b45d80f/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:04.730656 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b50ec3-8c60-4aee-b307-9e161191d898 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:04.730948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b50ec3-8c60-4aee-b307-9e161191d898 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:04.731265 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b50ec3-8c60-4aee-b307-9e161191d898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:04.752374 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-d5b50ec3-8c60-4aee-b307-9e161191d898 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:04.754224 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d5b50ec3-8c60-4aee-b307-9e161191d898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:04.798187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:04.820867 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1393/2772] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:04.830305 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cb958-6074-48d0-b0ef-fffde9f06ba2 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:04.832035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-bb6cb958-6074-48d0-b0ef-fffde9f06ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:04.890532 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cb958-6074-48d0-b0ef-fffde9f06ba2 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:04.894344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fb51c-c442-45d0-a2b4-d0f0215b97e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:04.895006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fb51c-c442-45d0-a2b4-d0f0215b97e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:04.895293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fb51c-c442-45d0-a2b4-d0f0215b97e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:04.895576 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fb51c-c442-45d0-a2b4-d0f0215b97e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:04.905465 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4fb51c-c442-45d0-a2b4-d0f0215b97e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:04.906982 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4fb51c-c442-45d0-a2b4-d0f0215b97e6 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:04.917795 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1380/2773] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:08.823428 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cc136-ab31-453a-9b26-bafdcbd9833b None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:08.823956 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cc136-ab31-453a-9b26-bafdcbd9833b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:08.824057 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cc136-ab31-453a-9b26-bafdcbd9833b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:08.862483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-443cc136-ab31-453a-9b26-bafdcbd9833b None None] MFA Rules not processed for user `52e9d0e3c2dd453eb4bfc2db6b45d80f`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:08.863754 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-443cc136-ab31-453a-9b26-bafdcbd9833b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:08.984622 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1394/2774] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:08.987760 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fb51c-c442-45d0-a2b4-d0f0215b97e6 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:09.006046 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fb51c-c442-45d0-a2b4-d0f0215b97e6 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:09.012425 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b21b8b-3fee-43ef-ba4a-92331580ce9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:09.013698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b21b8b-3fee-43ef-ba4a-92331580ce9e admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:09.014537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b21b8b-3fee-43ef-ba4a-92331580ce9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:09.015247 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b21b8b-3fee-43ef-ba4a-92331580ce9e admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:09.016170 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b21b8b-3fee-43ef-ba4a-92331580ce9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:09.018786 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b21b8b-3fee-43ef-ba4a-92331580ce9e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:09.069122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1381/2775] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:59:08 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:09.073394 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cc136-ab31-453a-9b26-bafdcbd9833b None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:09.087483 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cc136-ab31-453a-9b26-bafdcbd9833b None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:09.182917 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e22167-fb11-4f12-a7ad-20fcd196d782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:09.183744 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e22167-fb11-4f12-a7ad-20fcd196d782 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:09.183949 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e22167-fb11-4f12-a7ad-20fcd196d782 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:09.184248 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e22167-fb11-4f12-a7ad-20fcd196d782 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:09.185220 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e22167-fb11-4f12-a7ad-20fcd196d782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:09.187177 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e22167-fb11-4f12-a7ad-20fcd196d782 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:09.214392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1395/2776] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:59:09 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:59:09.218380 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b21b8b-3fee-43ef-ba4a-92331580ce9e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:09.230030 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b21b8b-3fee-43ef-ba4a-92331580ce9e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:09.237537 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964c6b5-ef2d-42b4-9769-ba834e961464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:09.238221 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964c6b5-ef2d-42b4-9769-ba834e961464 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:09.238510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964c6b5-ef2d-42b4-9769-ba834e961464 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:09.239074 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964c6b5-ef2d-42b4-9769-ba834e961464 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:09.239958 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964c6b5-ef2d-42b4-9769-ba834e961464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:09.242091 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964c6b5-ef2d-42b4-9769-ba834e961464 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:09.254256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1382/2777] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:59:09.260435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e22167-fb11-4f12-a7ad-20fcd196d782 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:09.271845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e22167-fb11-4f12-a7ad-20fcd196d782 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:09.290575 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbc28c-de45-42af-a64d-79ecf8ca99b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:09.291714 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbc28c-de45-42af-a64d-79ecf8ca99b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:09.292353 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbc28c-de45-42af-a64d-79ecf8ca99b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:09.292898 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbc28c-de45-42af-a64d-79ecf8ca99b0 admin admin] PATH_INFO: `/v3/projects/9b26a9a5df2b447cae8fc546e3e252c6/users/aa15c710c8ad42119203b1722750bf85/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:09.320175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbc28c-de45-42af-a64d-79ecf8ca99b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b26a9a5df2b447cae8fc546e3e252c6, user_id=aa15c710c8ad42119203b1722750bf85, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:09.322818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbc28c-de45-42af-a64d-79ecf8ca99b0 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:09.337250 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1396/2778] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:09 2022] PUT /identity/v3/projects/9b26a9a5df2b447cae8fc546e3e252c6/users/aa15c710c8ad42119203b1722750bf85/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:09.341877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964c6b5-ef2d-42b4-9769-ba834e961464 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:09.356255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964c6b5-ef2d-42b4-9769-ba834e961464 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:09.433234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4df74-1118-445e-9d5c-20dea1c1b394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:09.433922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4df74-1118-445e-9d5c-20dea1c1b394 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:09.442001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4df74-1118-445e-9d5c-20dea1c1b394 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:09.442488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4df74-1118-445e-9d5c-20dea1c1b394 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:09.443368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4df74-1118-445e-9d5c-20dea1c1b394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:09.446627 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4df74-1118-445e-9d5c-20dea1c1b394 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:09.448301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:09.464614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1383/2779] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:09.467512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbc28c-de45-42af-a64d-79ecf8ca99b0 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:09.485237 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbc28c-de45-42af-a64d-79ecf8ca99b0 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:09.491570 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec774f0-c724-441d-906f-f158c61d9149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:09.492655 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec774f0-c724-441d-906f-f158c61d9149 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:09.493017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec774f0-c724-441d-906f-f158c61d9149 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:09.493371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec774f0-c724-441d-906f-f158c61d9149 admin admin] PATH_INFO: `/v3/projects/9b26a9a5df2b447cae8fc546e3e252c6/users/aa15c710c8ad42119203b1722750bf85/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:09.495830 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec774f0-c724-441d-906f-f158c61d9149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b26a9a5df2b447cae8fc546e3e252c6, user_id=aa15c710c8ad42119203b1722750bf85, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:09.498639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec774f0-c724-441d-906f-f158c61d9149 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:09.510631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1397/2780] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:09 2022] PUT /identity/v3/projects/9b26a9a5df2b447cae8fc546e3e252c6/users/aa15c710c8ad42119203b1722750bf85/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:13.871636 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4df74-1118-445e-9d5c-20dea1c1b394 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:13.873683 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-65e4df74-1118-445e-9d5c-20dea1c1b394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:13.902318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4df74-1118-445e-9d5c-20dea1c1b394 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:14.006186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:14.007508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:14.007508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:14.007822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] PATH_INFO: `/v3/users/811d782c18e44b6c88381978a86f53ca` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:14.009132 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=811d782c18e44b6c88381978a86f53ca)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:14.011234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:14.107699 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '811d782c18e44b6c88381978a86f53ca'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:14.119881 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '811d782c18e44b6c88381978a86f53ca'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:14.138663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '811d782c18e44b6c88381978a86f53ca'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:14.165254 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '811d782c18e44b6c88381978a86f53ca'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:14.166169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '811d782c18e44b6c88381978a86f53ca'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:14.181852 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1384/2781] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:59:13 2022] DELETE /identity/v3/users/811d782c18e44b6c88381978a86f53ca => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:14.183693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf64cbe-5e9c-4da3-abae-250afc62b19b None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:14.184207 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf64cbe-5e9c-4da3-abae-250afc62b19b None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:14.184447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf64cbe-5e9c-4da3-abae-250afc62b19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:14.208595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-fbf64cbe-5e9c-4da3-abae-250afc62b19b None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:14.210921 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fbf64cbe-5e9c-4da3-abae-250afc62b19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:14.284859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:14.390592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1398/2782] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:14.401847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:14.403307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:14.479033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde7afc-6a55-4fa5-bb84-f760f83a1ecb admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:14.483513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522100ce-0766-4fc4-ade3-3476340c5981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:14.484335 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522100ce-0766-4fc4-ade3-3476340c5981 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:14.484762 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522100ce-0766-4fc4-ade3-3476340c5981 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:14.485233 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522100ce-0766-4fc4-ade3-3476340c5981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:14.495893 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522100ce-0766-4fc4-ade3-3476340c5981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:14.497456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522100ce-0766-4fc4-ade3-3476340c5981 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:14.511492 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1385/2783] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:14.544730 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a92ed-d6a4-4336-aaab-e1baf54eddbe None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:14.545102 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a92ed-d6a4-4336-aaab-e1baf54eddbe None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:14.545891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a92ed-d6a4-4336-aaab-e1baf54eddbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:14.583081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-f36a92ed-d6a4-4336-aaab-e1baf54eddbe None None] MFA Rules not processed for user `aa15c710c8ad42119203b1722750bf85`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:14.585326 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f36a92ed-d6a4-4336-aaab-e1baf54eddbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:14.681228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1399/2784] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:14.704039 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522100ce-0766-4fc4-ade3-3476340c5981 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:14.709055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-522100ce-0766-4fc4-ade3-3476340c5981 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:14.738424 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:14.804466 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522100ce-0766-4fc4-ade3-3476340c5981 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:14.817016 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563363e-b3a4-49b7-b2df-67f76bceed90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:14.818189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563363e-b3a4-49b7-b2df-67f76bceed90 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:14.818189 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563363e-b3a4-49b7-b2df-67f76bceed90 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:14.818437 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563363e-b3a4-49b7-b2df-67f76bceed90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:14.820409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c563363e-b3a4-49b7-b2df-67f76bceed90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:14.849643 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a92ed-d6a4-4336-aaab-e1baf54eddbe None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:14.851179 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f36a92ed-d6a4-4336-aaab-e1baf54eddbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:14.868321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a92ed-d6a4-4336-aaab-e1baf54eddbe None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:14.875537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:14.878680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:14.878680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:14.878680 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] PATH_INFO: `/v3/projects/c61846255f854229baeeba0b72eaa477` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:14.881037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61846255f854229baeeba0b72eaa477)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:14.883078 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:14.913450 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563363e-b3a4-49b7-b2df-67f76bceed90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:14.915318 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563363e-b3a4-49b7-b2df-67f76bceed90 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:14.927763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1386/2785] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:14.952133 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61846255f854229baeeba0b72eaa477', 'request_id': 'req-c30092f7-a0cf-4ce0-9349-ad35774e966e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:14.966924 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61846255f854229baeeba0b72eaa477', 'request_id': 'req-c30092f7-a0cf-4ce0-9349-ad35774e966e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:14.978664 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61846255f854229baeeba0b72eaa477', 'request_id': 'req-c30092f7-a0cf-4ce0-9349-ad35774e966e'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:14.999703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] The token cache is being invalidate because project c61846255f854229baeeba0b72eaa477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:59:15.000005 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:15.000863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:15.004624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1400/2786] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:59:14 2022] DELETE /identity/v3/projects/c61846255f854229baeeba0b72eaa477 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:16.476799 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dafbf8-c82b-45f5-98c3-e3eb55e7e7c5 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:16.477266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dafbf8-c82b-45f5-98c3-e3eb55e7e7c5 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:16.477368 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dafbf8-c82b-45f5-98c3-e3eb55e7e7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:16.498848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-b9dafbf8-c82b-45f5-98c3-e3eb55e7e7c5 None None] MFA Rules not processed for user `52e9d0e3c2dd453eb4bfc2db6b45d80f`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:16.500078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b9dafbf8-c82b-45f5-98c3-e3eb55e7e7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:16.565966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1387/2787] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:24.996049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:26.558052 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=73 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:35.004939 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:35.006370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:35.065114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30092f7-a0cf-4ce0-9349-ad35774e966e admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:35.068790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09cfb4-de4f-4795-9b3d-2288525f1da7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:35.069451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09cfb4-de4f-4795-9b3d-2288525f1da7 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:35.069738 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09cfb4-de4f-4795-9b3d-2288525f1da7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:35.070016 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09cfb4-de4f-4795-9b3d-2288525f1da7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:35.071938 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-bd09cfb4-de4f-4795-9b3d-2288525f1da7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:35.156132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09cfb4-de4f-4795-9b3d-2288525f1da7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:35.157620 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09cfb4-de4f-4795-9b3d-2288525f1da7 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:35.177705 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1401/2788] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:38.793422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dafbf8-c82b-45f5-98c3-e3eb55e7e7c5 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:38.795801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b9dafbf8-c82b-45f5-98c3-e3eb55e7e7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:38.897606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dafbf8-c82b-45f5-98c3-e3eb55e7e7c5 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:38.906168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7019442-a584-452e-ba35-3ca58ace0a51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:38.906887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7019442-a584-452e-ba35-3ca58ace0a51 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:38.907158 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7019442-a584-452e-ba35-3ca58ace0a51 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:38.907439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7019442-a584-452e-ba35-3ca58ace0a51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:38.910092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b7019442-a584-452e-ba35-3ca58ace0a51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:38.924850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7019442-a584-452e-ba35-3ca58ace0a51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:38.927004 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7019442-a584-452e-ba35-3ca58ace0a51 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:38.944801 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1388/2789] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:38.987370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09cfb4-de4f-4795-9b3d-2288525f1da7 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:38.989291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-bd09cfb4-de4f-4795-9b3d-2288525f1da7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:39.077868 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09cfb4-de4f-4795-9b3d-2288525f1da7 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:39.085746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f87e0-e84f-44de-85de-96e7bcdfc528 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:39.086411 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f87e0-e84f-44de-85de-96e7bcdfc528 service placement] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:39.086697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f87e0-e84f-44de-85de-96e7bcdfc528 service placement] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:39.086978 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f87e0-e84f-44de-85de-96e7bcdfc528 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:39.088856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-194f87e0-e84f-44de-85de-96e7bcdfc528 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:39.102158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f87e0-e84f-44de-85de-96e7bcdfc528 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:39.103795 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f87e0-e84f-44de-85de-96e7bcdfc528 service placement] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:39.121156 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1402/2790] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:40.605742 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7019442-a584-452e-ba35-3ca58ace0a51 service nova] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:40.607307 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b7019442-a584-452e-ba35-3ca58ace0a51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:40.667139 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7019442-a584-452e-ba35-3ca58ace0a51 service nova] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:40.671312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dbde7-151e-4190-8893-15cbb2aafe7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:40.672127 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dbde7-151e-4190-8893-15cbb2aafe7e service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:40.672421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dbde7-151e-4190-8893-15cbb2aafe7e service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:40.672732 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dbde7-151e-4190-8893-15cbb2aafe7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:40.674749 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b00dbde7-151e-4190-8893-15cbb2aafe7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:40.685474 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dbde7-151e-4190-8893-15cbb2aafe7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:40.686928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dbde7-151e-4190-8893-15cbb2aafe7e service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:40.695951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1389/2791] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:43.447376 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f87e0-e84f-44de-85de-96e7bcdfc528 service placement] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:43.449142 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-194f87e0-e84f-44de-85de-96e7bcdfc528 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:43.466692 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f87e0-e84f-44de-85de-96e7bcdfc528 service placement] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:43.471054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980791e-e890-4542-9d57-8f9c600ea421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:43.471755 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980791e-e890-4542-9d57-8f9c600ea421 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:43.472071 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980791e-e890-4542-9d57-8f9c600ea421 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:43.472352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980791e-e890-4542-9d57-8f9c600ea421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:43.474251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4980791e-e890-4542-9d57-8f9c600ea421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:43.501034 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=100 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:43.564426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980791e-e890-4542-9d57-8f9c600ea421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:43.567084 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980791e-e890-4542-9d57-8f9c600ea421 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:43.578828 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1403/2792] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:46.800694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dbde7-151e-4190-8893-15cbb2aafe7e service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:46.802503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b00dbde7-151e-4190-8893-15cbb2aafe7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:46.819720 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dbde7-151e-4190-8893-15cbb2aafe7e service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:46.824584 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aedaee-1323-4623-ba54-79cac6d3f42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:46.825523 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aedaee-1323-4623-ba54-79cac6d3f42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:46.827157 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aedaee-1323-4623-ba54-79cac6d3f42d admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:46.827157 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aedaee-1323-4623-ba54-79cac6d3f42d admin admin] PATH_INFO: `/v3/users/d3f0b30685e94f83883d682731644a89` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:46.827887 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aedaee-1323-4623-ba54-79cac6d3f42d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3f0b30685e94f83883d682731644a89)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:46.829864 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aedaee-1323-4623-ba54-79cac6d3f42d admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:46.878683 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-63aedaee-1323-4623-ba54-79cac6d3f42d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3f0b30685e94f83883d682731644a89'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:46.886289 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-63aedaee-1323-4623-ba54-79cac6d3f42d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3f0b30685e94f83883d682731644a89'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:46.898902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-63aedaee-1323-4623-ba54-79cac6d3f42d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3f0b30685e94f83883d682731644a89'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:46.905873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-63aedaee-1323-4623-ba54-79cac6d3f42d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3f0b30685e94f83883d682731644a89'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:46.906639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-63aedaee-1323-4623-ba54-79cac6d3f42d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3f0b30685e94f83883d682731644a89'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:46.910894 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:46.918141 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1390/2793] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:59:46 2022] DELETE /identity/v3/users/d3f0b30685e94f83883d682731644a89 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:47.104221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980791e-e890-4542-9d57-8f9c600ea421 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:47.116146 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980791e-e890-4542-9d57-8f9c600ea421 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:47.179679 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:47.180563 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:47.180881 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:47.181164 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f admin admin] PATH_INFO: `/v3/projects/f3491b85dc4543c88f500828b2df1d2d` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:47.182443 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3491b85dc4543c88f500828b2df1d2d)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:47.184411 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:47.244896 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3491b85dc4543c88f500828b2df1d2d', 'request_id': 'req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:47.255346 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3491b85dc4543c88f500828b2df1d2d', 'request_id': 'req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:47.261582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3491b85dc4543c88f500828b2df1d2d', 'request_id': 'req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:47.275347 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f admin admin] The token cache is being invalidate because project f3491b85dc4543c88f500828b2df1d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:59:47.275640 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:47.276328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-ddf808a3-f59e-4e00-baa5-199fa41e2c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:47.278140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1404/2794] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:59:47 2022] DELETE /identity/v3/projects/f3491b85dc4543c88f500828b2df1d2d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:47.283183 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0e600-4fbf-4672-99d1-5e9b276c3f45 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:47.283602 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0e600-4fbf-4672-99d1-5e9b276c3f45 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:47.283946 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0e600-4fbf-4672-99d1-5e9b276c3f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:47.306085 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-04e0e600-4fbf-4672-99d1-5e9b276c3f45 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:47.307427 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-04e0e600-4fbf-4672-99d1-5e9b276c3f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:47.372855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1391/2795] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:47.381648 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bae659-13a0-47f8-92a0-4687f53e9454 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:47.381942 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bae659-13a0-47f8-92a0-4687f53e9454 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:47.382228 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bae659-13a0-47f8-92a0-4687f53e9454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:47.402737 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-c9bae659-13a0-47f8-92a0-4687f53e9454 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:47.403988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-c9bae659-13a0-47f8-92a0-4687f53e9454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:47.412516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1405/2796] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:59:47.416942 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0e600-4fbf-4672-99d1-5e9b276c3f45 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:47.427922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0e600-4fbf-4672-99d1-5e9b276c3f45 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:47.433772 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24467f5c-d58a-4406-b45b-08f70c5ac0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:47.434481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24467f5c-d58a-4406-b45b-08f70c5ac0ef admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:47.434869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24467f5c-d58a-4406-b45b-08f70c5ac0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:47.435145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24467f5c-d58a-4406-b45b-08f70c5ac0ef admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:47.435892 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24467f5c-d58a-4406-b45b-08f70c5ac0ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:59:47.436192 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24467f5c-d58a-4406-b45b-08f70c5ac0ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:47.437995 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24467f5c-d58a-4406-b45b-08f70c5ac0ef admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:47.451578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1392/2797] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:59:47.456899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bae659-13a0-47f8-92a0-4687f53e9454 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:47.467318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bae659-13a0-47f8-92a0-4687f53e9454 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:47.472300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e0a34-83e3-43e2-98c8-02337242918f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:47.473007 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e0a34-83e3-43e2-98c8-02337242918f admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:47.473289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e0a34-83e3-43e2-98c8-02337242918f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:47.473564 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e0a34-83e3-43e2-98c8-02337242918f admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:47.474284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e0a34-83e3-43e2-98c8-02337242918f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:47.476634 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e0a34-83e3-43e2-98c8-02337242918f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:47.513311 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1406/2798] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:59:47 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:47.518014 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24467f5c-d58a-4406-b45b-08f70c5ac0ef admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:47.532206 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24467f5c-d58a-4406-b45b-08f70c5ac0ef admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:47.610193 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa3a1-803e-4004-a803-b944c4fcab2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:47.611059 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa3a1-803e-4004-a803-b944c4fcab2f admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:47.611353 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa3a1-803e-4004-a803-b944c4fcab2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:47.611639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa3a1-803e-4004-a803-b944c4fcab2f admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:47.612993 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa3a1-803e-4004-a803-b944c4fcab2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:47.614925 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa3a1-803e-4004-a803-b944c4fcab2f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:47.642431 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1393/2799] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:59:47 2022] POST /identity/v3/users => generated 283 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:47.643625 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e0a34-83e3-43e2-98c8-02337242918f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:47.654366 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e0a34-83e3-43e2-98c8-02337242918f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:47.659633 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b813889-7561-4185-b0fe-9458c9487423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:47.660305 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b813889-7561-4185-b0fe-9458c9487423 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:47.660616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b813889-7561-4185-b0fe-9458c9487423 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:47.660904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b813889-7561-4185-b0fe-9458c9487423 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:47.661643 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b813889-7561-4185-b0fe-9458c9487423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:47.663321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b813889-7561-4185-b0fe-9458c9487423 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:47.669344 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:47.678545 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1407/2800] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 08:59:47.679927 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa3a1-803e-4004-a803-b944c4fcab2f admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:47.690271 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa3a1-803e-4004-a803-b944c4fcab2f admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:47.695002 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b78537-2c58-4d91-8497-baf77063dc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:47.695660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b78537-2c58-4d91-8497-baf77063dc8f admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:47.695943 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b78537-2c58-4d91-8497-baf77063dc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:47.696227 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b78537-2c58-4d91-8497-baf77063dc8f admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:47.696998 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b78537-2c58-4d91-8497-baf77063dc8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:47.698599 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b78537-2c58-4d91-8497-baf77063dc8f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:47.710696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1394/2801] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:47.715158 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b813889-7561-4185-b0fe-9458c9487423 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:47.727062 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b813889-7561-4185-b0fe-9458c9487423 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:47.732042 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7cdea-8c36-4bc0-82ae-a602af20912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:47.732980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7cdea-8c36-4bc0-82ae-a602af20912c admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:47.733265 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7cdea-8c36-4bc0-82ae-a602af20912c admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:47.733609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7cdea-8c36-4bc0-82ae-a602af20912c admin admin] PATH_INFO: `/v3/projects/85ed009356964c7387abd374f8d4d0ee/users/588765004dc444c7bcd88b969caa4167/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:47.752392 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7cdea-8c36-4bc0-82ae-a602af20912c admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ed009356964c7387abd374f8d4d0ee, user_id=588765004dc444c7bcd88b969caa4167, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:47.755202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7cdea-8c36-4bc0-82ae-a602af20912c admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:47.772274 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1408/2802] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:47 2022] PUT /identity/v3/projects/85ed009356964c7387abd374f8d4d0ee/users/588765004dc444c7bcd88b969caa4167/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:47.786558 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2245e2-32c6-4359-a029-75def6061707 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:47.786844 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2245e2-32c6-4359-a029-75def6061707 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:47.787121 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2245e2-32c6-4359-a029-75def6061707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:47.819395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-1a2245e2-32c6-4359-a029-75def6061707 None None] MFA Rules not processed for user `588765004dc444c7bcd88b969caa4167`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:47.821163 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-1a2245e2-32c6-4359-a029-75def6061707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:47.877857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:47.888726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1395/2803] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:59:47.892327 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7cdea-8c36-4bc0-82ae-a602af20912c admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:47.961990 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7cdea-8c36-4bc0-82ae-a602af20912c admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:47.966806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d5f31-fc32-4e28-bc88-38aabf772d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:47.967477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d5f31-fc32-4e28-bc88-38aabf772d8a admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:47.967765 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d5f31-fc32-4e28-bc88-38aabf772d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:47.968049 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d5f31-fc32-4e28-bc88-38aabf772d8a admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:47.968797 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d5f31-fc32-4e28-bc88-38aabf772d8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:47.970721 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d5f31-fc32-4e28-bc88-38aabf772d8a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:47.999843 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1409/2804] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:59:47 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:48.003940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2245e2-32c6-4359-a029-75def6061707 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:48.015065 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2245e2-32c6-4359-a029-75def6061707 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:48.073848 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf10a9-040c-4e6d-aab5-683de17f5783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:48.074531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf10a9-040c-4e6d-aab5-683de17f5783 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:48.074818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf10a9-040c-4e6d-aab5-683de17f5783 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:48.075097 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf10a9-040c-4e6d-aab5-683de17f5783 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:48.075812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf10a9-040c-4e6d-aab5-683de17f5783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:48.077771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf10a9-040c-4e6d-aab5-683de17f5783 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:48.101741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1396/2805] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:59:48 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 08:59:48.111759 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d5f31-fc32-4e28-bc88-38aabf772d8a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:48.122332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d5f31-fc32-4e28-bc88-38aabf772d8a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:48.129107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945d2a1-f18d-4dcc-b379-fbb7667e2ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:48.130101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945d2a1-f18d-4dcc-b379-fbb7667e2ffa admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:48.130528 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945d2a1-f18d-4dcc-b379-fbb7667e2ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:48.130950 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945d2a1-f18d-4dcc-b379-fbb7667e2ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:48.132090 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945d2a1-f18d-4dcc-b379-fbb7667e2ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:48.134715 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945d2a1-f18d-4dcc-b379-fbb7667e2ffa admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:48.155717 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1410/2806] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:48.156923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf10a9-040c-4e6d-aab5-683de17f5783 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:48.173566 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf10a9-040c-4e6d-aab5-683de17f5783 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:48.178667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a3a3-ad61-4d97-b9fe-fc224ec204b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:48.179570 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a3a3-ad61-4d97-b9fe-fc224ec204b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:48.179857 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a3a3-ad61-4d97-b9fe-fc224ec204b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:48.180138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a3a3-ad61-4d97-b9fe-fc224ec204b4 admin admin] PATH_INFO: `/v3/projects/e4e2d39358004481a0992e42f98eaa3b/users/4daaa6bc9d6a44f0b499698ef650f3b9/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:48.196915 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82a3a3-ad61-4d97-b9fe-fc224ec204b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e2d39358004481a0992e42f98eaa3b, user_id=4daaa6bc9d6a44f0b499698ef650f3b9, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:48.199581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82a3a3-ad61-4d97-b9fe-fc224ec204b4 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:48.213951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1397/2807] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:48 2022] PUT /identity/v3/projects/e4e2d39358004481a0992e42f98eaa3b/users/4daaa6bc9d6a44f0b499698ef650f3b9/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:59:48.217836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945d2a1-f18d-4dcc-b379-fbb7667e2ffa admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:48.229016 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945d2a1-f18d-4dcc-b379-fbb7667e2ffa admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:48.289492 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167e7d8-91be-416f-90de-75f5def57fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:48.290195 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e7d8-91be-416f-90de-75f5def57fcc admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:48.290485 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e7d8-91be-416f-90de-75f5def57fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:48.290788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e7d8-91be-416f-90de-75f5def57fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:48.291523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167e7d8-91be-416f-90de-75f5def57fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:48.293207 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167e7d8-91be-416f-90de-75f5def57fcc admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:48.307862 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1411/2808] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:48.309114 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a3a3-ad61-4d97-b9fe-fc224ec204b4 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:48.319751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a3a3-ad61-4d97-b9fe-fc224ec204b4 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:48.324600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6f47b-2f4a-4c0c-b4ab-fbb42f4acb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:48.325484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6f47b-2f4a-4c0c-b4ab-fbb42f4acb03 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:48.325763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6f47b-2f4a-4c0c-b4ab-fbb42f4acb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:48.326037 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6f47b-2f4a-4c0c-b4ab-fbb42f4acb03 admin admin] PATH_INFO: `/v3/projects/e4e2d39358004481a0992e42f98eaa3b/users/4daaa6bc9d6a44f0b499698ef650f3b9/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:48.328083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6f47b-2f4a-4c0c-b4ab-fbb42f4acb03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e2d39358004481a0992e42f98eaa3b, user_id=4daaa6bc9d6a44f0b499698ef650f3b9, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:48.330780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6f47b-2f4a-4c0c-b4ab-fbb42f4acb03 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:48.342337 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1398/2809] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:48 2022] PUT /identity/v3/projects/e4e2d39358004481a0992e42f98eaa3b/users/4daaa6bc9d6a44f0b499698ef650f3b9/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:48.354435 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5105-2e7c-44a1-be8c-25fb20552619 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:48.354818 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5105-2e7c-44a1-be8c-25fb20552619 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:48.355202 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5105-2e7c-44a1-be8c-25fb20552619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:48.387817 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-ac1f5105-2e7c-44a1-be8c-25fb20552619 None None] MFA Rules not processed for user `4daaa6bc9d6a44f0b499698ef650f3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:48.389057 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-ac1f5105-2e7c-44a1-be8c-25fb20552619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:48.429599 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:48.474899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1412/2810] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:48 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:59:48.492455 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6f47b-2f4a-4c0c-b4ab-fbb42f4acb03 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:48.494306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f6d6f47b-2f4a-4c0c-b4ab-fbb42f4acb03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:48.556203 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6f47b-2f4a-4c0c-b4ab-fbb42f4acb03 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:48.559919 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a09b0e-fa36-4604-9b74-0db0e7c9bc4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:48.560588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a09b0e-fa36-4604-9b74-0db0e7c9bc4f service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:48.560884 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a09b0e-fa36-4604-9b74-0db0e7c9bc4f service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:48.561157 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a09b0e-fa36-4604-9b74-0db0e7c9bc4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:48.571808 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a09b0e-fa36-4604-9b74-0db0e7c9bc4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:48.573322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a09b0e-fa36-4604-9b74-0db0e7c9bc4f service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:48.584079 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1399/2811] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:48 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:49.030601 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5105-2e7c-44a1-be8c-25fb20552619 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:49.094863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5105-2e7c-44a1-be8c-25fb20552619 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:49.101853 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:49.102740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:49.103024 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:49.103289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] PATH_INFO: `/v3/users/588765004dc444c7bcd88b969caa4167` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:49.104552 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=588765004dc444c7bcd88b969caa4167)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:49.106516 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:49.158028 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '588765004dc444c7bcd88b969caa4167'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.158706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '588765004dc444c7bcd88b969caa4167'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.167558 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '588765004dc444c7bcd88b969caa4167'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.180337 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '588765004dc444c7bcd88b969caa4167'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.187240 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '588765004dc444c7bcd88b969caa4167'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.195949 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1413/2812] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:59:49 2022] DELETE /identity/v3/users/588765004dc444c7bcd88b969caa4167 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:49.200607 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf6549-bf55-4135-ba1c-4c77cd4b3c2a None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:49.201098 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf6549-bf55-4135-ba1c-4c77cd4b3c2a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:49.201225 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf6549-bf55-4135-ba1c-4c77cd4b3c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:49.223358 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-4eaf6549-bf55-4135-ba1c-4c77cd4b3c2a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:49.227458 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4eaf6549-bf55-4135-ba1c-4c77cd4b3c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:49.269741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:49.285313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1400/2813] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:49.295501 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:49.297035 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:49.356858 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202eb45-c5d9-47c0-a1ff-da058c05c973 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:49.360609 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc727ac6-8bf6-4ebe-98fe-b35067e6e515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:49.361276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc727ac6-8bf6-4ebe-98fe-b35067e6e515 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:49.361566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc727ac6-8bf6-4ebe-98fe-b35067e6e515 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:49.361845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc727ac6-8bf6-4ebe-98fe-b35067e6e515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:49.368661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:49.373020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc727ac6-8bf6-4ebe-98fe-b35067e6e515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:49.374836 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc727ac6-8bf6-4ebe-98fe-b35067e6e515 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:49.386595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1414/2814] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:49.680729 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf6549-bf55-4135-ba1c-4c77cd4b3c2a None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:49.682193 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4eaf6549-bf55-4135-ba1c-4c77cd4b3c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:49.697453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf6549-bf55-4135-ba1c-4c77cd4b3c2a None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:49.702856 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:49.703819 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:49.704097 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:49.704386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] PATH_INFO: `/v3/projects/85ed009356964c7387abd374f8d4d0ee` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:49.706614 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] RBAC: Authorizing `identity:delete_project(project_id=85ed009356964c7387abd374f8d4d0ee)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:49.709109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:49.762680 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85ed009356964c7387abd374f8d4d0ee', 'request_id': 'req-f1abc408-5b78-4293-8454-58ddfdb7199c'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.771763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85ed009356964c7387abd374f8d4d0ee', 'request_id': 'req-f1abc408-5b78-4293-8454-58ddfdb7199c'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.777944 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85ed009356964c7387abd374f8d4d0ee', 'request_id': 'req-f1abc408-5b78-4293-8454-58ddfdb7199c'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.793683 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] The token cache is being invalidate because project 85ed009356964c7387abd374f8d4d0ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:59:49.793880 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.794541 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.796083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1401/2815] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:59:49 2022] DELETE /identity/v3/projects/85ed009356964c7387abd374f8d4d0ee => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:49.800277 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc727ac6-8bf6-4ebe-98fe-b35067e6e515 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:49.810182 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc727ac6-8bf6-4ebe-98fe-b35067e6e515 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:49.867290 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e0af4-d200-4f09-92b5-16cbcd826e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:49.868172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e0af4-d200-4f09-92b5-16cbcd826e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:49.868462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e0af4-d200-4f09-92b5-16cbcd826e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:49.868865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e0af4-d200-4f09-92b5-16cbcd826e99 admin admin] PATH_INFO: `/v3/users/4daaa6bc9d6a44f0b499698ef650f3b9` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:49.870844 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e0af4-d200-4f09-92b5-16cbcd826e99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4daaa6bc9d6a44f0b499698ef650f3b9)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:49.873998 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e0af4-d200-4f09-92b5-16cbcd826e99 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:49.925850 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f05e0af4-d200-4f09-92b5-16cbcd826e99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4daaa6bc9d6a44f0b499698ef650f3b9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.926531 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f05e0af4-d200-4f09-92b5-16cbcd826e99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4daaa6bc9d6a44f0b499698ef650f3b9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.934412 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f05e0af4-d200-4f09-92b5-16cbcd826e99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4daaa6bc9d6a44f0b499698ef650f3b9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.949213 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f05e0af4-d200-4f09-92b5-16cbcd826e99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4daaa6bc9d6a44f0b499698ef650f3b9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.957084 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-f05e0af4-d200-4f09-92b5-16cbcd826e99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4daaa6bc9d6a44f0b499698ef650f3b9'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:49.963252 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1415/2816] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:59:49 2022] DELETE /identity/v3/users/4daaa6bc9d6a44f0b499698ef650f3b9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:50.239452 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:50.240997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:50.255022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abc408-5b78-4293-8454-58ddfdb7199c admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:50.312831 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0d0eb-77eb-42a9-8577-1fe676397160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:50.313624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0d0eb-77eb-42a9-8577-1fe676397160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:50.313905 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0d0eb-77eb-42a9-8577-1fe676397160 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:50.314176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0d0eb-77eb-42a9-8577-1fe676397160 admin admin] PATH_INFO: `/v3/projects/e4e2d39358004481a0992e42f98eaa3b` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:50.315921 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0d0eb-77eb-42a9-8577-1fe676397160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4e2d39358004481a0992e42f98eaa3b)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:50.317840 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0d0eb-77eb-42a9-8577-1fe676397160 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:50.364805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4db0d0eb-77eb-42a9-8577-1fe676397160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4e2d39358004481a0992e42f98eaa3b', 'request_id': 'req-4db0d0eb-77eb-42a9-8577-1fe676397160'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:50.373979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4db0d0eb-77eb-42a9-8577-1fe676397160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4e2d39358004481a0992e42f98eaa3b', 'request_id': 'req-4db0d0eb-77eb-42a9-8577-1fe676397160'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:50.380604 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4db0d0eb-77eb-42a9-8577-1fe676397160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4e2d39358004481a0992e42f98eaa3b', 'request_id': 'req-4db0d0eb-77eb-42a9-8577-1fe676397160'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:50.392503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4db0d0eb-77eb-42a9-8577-1fe676397160 admin admin] The token cache is being invalidate because project e4e2d39358004481a0992e42f98eaa3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:59:50.392917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4db0d0eb-77eb-42a9-8577-1fe676397160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:50.393513 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-4db0d0eb-77eb-42a9-8577-1fe676397160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:50.395055 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1402/2817] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:59:50 2022] DELETE /identity/v3/projects/e4e2d39358004481a0992e42f98eaa3b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:50.399928 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6872b4-d228-4860-aef0-0ecf559edfa7 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:50.400456 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6872b4-d228-4860-aef0-0ecf559edfa7 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:50.400758 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6872b4-d228-4860-aef0-0ecf559edfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:50.421187 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-7c6872b4-d228-4860-aef0-0ecf559edfa7 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:50.422420 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7c6872b4-d228-4860-aef0-0ecf559edfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:50.482945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1416/2818] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:50.491403 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04567e1e-158f-4de9-bf74-894f7f7b2830 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:50.491695 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04567e1e-158f-4de9-bf74-894f7f7b2830 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:50.491973 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04567e1e-158f-4de9-bf74-894f7f7b2830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:50.501266 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:50.518475 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-04567e1e-158f-4de9-bf74-894f7f7b2830 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:50.520663 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-04567e1e-158f-4de9-bf74-894f7f7b2830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:50.529862 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1403/2819] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:50.534229 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6872b4-d228-4860-aef0-0ecf559edfa7 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:50.544339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6872b4-d228-4860-aef0-0ecf559edfa7 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:50.549065 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819933b4-444e-461c-8ac2-f5e1cf8e628a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:50.552030 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819933b4-444e-461c-8ac2-f5e1cf8e628a admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:50.552030 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819933b4-444e-461c-8ac2-f5e1cf8e628a admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:50.552030 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819933b4-444e-461c-8ac2-f5e1cf8e628a admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:50.552030 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819933b4-444e-461c-8ac2-f5e1cf8e628a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:59:50.552030 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819933b4-444e-461c-8ac2-f5e1cf8e628a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:50.553438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819933b4-444e-461c-8ac2-f5e1cf8e628a admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:50.564819 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:50.569178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1417/2820] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:59:50.573867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04567e1e-158f-4de9-bf74-894f7f7b2830 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:50.583356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04567e1e-158f-4de9-bf74-894f7f7b2830 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:50.588033 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0b531-a352-4924-be3f-17584b7a561f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:50.588724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0b531-a352-4924-be3f-17584b7a561f admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:50.589006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0b531-a352-4924-be3f-17584b7a561f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:50.589281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0b531-a352-4924-be3f-17584b7a561f admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:50.590010 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0b531-a352-4924-be3f-17584b7a561f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:50.591899 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0b531-a352-4924-be3f-17584b7a561f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:50.622923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1404/2821] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:59:50 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:50.627074 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819933b4-444e-461c-8ac2-f5e1cf8e628a admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:50.638697 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819933b4-444e-461c-8ac2-f5e1cf8e628a admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:50.696895 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574186f6-6980-44c1-a6c6-13669ffb79de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:50.697581 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574186f6-6980-44c1-a6c6-13669ffb79de admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:50.697863 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574186f6-6980-44c1-a6c6-13669ffb79de admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:50.698140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574186f6-6980-44c1-a6c6-13669ffb79de admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:50.698856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574186f6-6980-44c1-a6c6-13669ffb79de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:50.700807 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574186f6-6980-44c1-a6c6-13669ffb79de admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:50.723451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1418/2822] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:59:50 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:50.727517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0b531-a352-4924-be3f-17584b7a561f admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:50.738741 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0b531-a352-4924-be3f-17584b7a561f admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:50.743595 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94995d7a-0cb0-4fd4-8e3f-5f878180c434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:50.744300 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94995d7a-0cb0-4fd4-8e3f-5f878180c434 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:50.744617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94995d7a-0cb0-4fd4-8e3f-5f878180c434 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:50.744888 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94995d7a-0cb0-4fd4-8e3f-5f878180c434 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:50.745880 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94995d7a-0cb0-4fd4-8e3f-5f878180c434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:50.747476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94995d7a-0cb0-4fd4-8e3f-5f878180c434 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:50.758340 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1405/2823] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:50.762491 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574186f6-6980-44c1-a6c6-13669ffb79de admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:50.772814 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574186f6-6980-44c1-a6c6-13669ffb79de admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:50.777577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70faede-f513-461b-8a43-1fb3964ed74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:50.778256 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70faede-f513-461b-8a43-1fb3964ed74d admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:50.778543 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70faede-f513-461b-8a43-1fb3964ed74d admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:50.778824 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70faede-f513-461b-8a43-1fb3964ed74d admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:50.779569 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70faede-f513-461b-8a43-1fb3964ed74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:50.781223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70faede-f513-461b-8a43-1fb3964ed74d admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:50.793592 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1419/2824] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:50.797606 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94995d7a-0cb0-4fd4-8e3f-5f878180c434 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:50.809188 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94995d7a-0cb0-4fd4-8e3f-5f878180c434 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:50.814211 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74eb7f-2596-4805-9eeb-41a2f4fbcbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:50.815234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74eb7f-2596-4805-9eeb-41a2f4fbcbb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:50.815444 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74eb7f-2596-4805-9eeb-41a2f4fbcbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:50.815723 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74eb7f-2596-4805-9eeb-41a2f4fbcbb8 admin admin] PATH_INFO: `/v3/projects/bdf850fce06f4d558da0b4bee7dfe728/users/ce7d85a5a9474e3bbb999e9ae64df152/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:50.832061 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74eb7f-2596-4805-9eeb-41a2f4fbcbb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf850fce06f4d558da0b4bee7dfe728, user_id=ce7d85a5a9474e3bbb999e9ae64df152, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:50.834765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74eb7f-2596-4805-9eeb-41a2f4fbcbb8 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:50.853823 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1406/2825] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:50 2022] PUT /identity/v3/projects/bdf850fce06f4d558da0b4bee7dfe728/users/ce7d85a5a9474e3bbb999e9ae64df152/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:59:50.876963 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c28ba-4e93-4330-b03d-13e91e7d0a92 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:50.877548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c28ba-4e93-4330-b03d-13e91e7d0a92 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:50.877548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c28ba-4e93-4330-b03d-13e91e7d0a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:50.909773 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-066c28ba-4e93-4330-b03d-13e91e7d0a92 None None] MFA Rules not processed for user `ce7d85a5a9474e3bbb999e9ae64df152`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:50.911457 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-066c28ba-4e93-4330-b03d-13e91e7d0a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:50.981185 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1420/2826] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 08:59:50.985618 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74eb7f-2596-4805-9eeb-41a2f4fbcbb8 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:51.050056 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74eb7f-2596-4805-9eeb-41a2f4fbcbb8 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:51.054812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a12636-fd36-41b8-8b13-109b21a4bda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:51.055481 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a12636-fd36-41b8-8b13-109b21a4bda3 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:51.055771 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a12636-fd36-41b8-8b13-109b21a4bda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:51.056047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a12636-fd36-41b8-8b13-109b21a4bda3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:51.056778 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a12636-fd36-41b8-8b13-109b21a4bda3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:51.058686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a12636-fd36-41b8-8b13-109b21a4bda3 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:51.085743 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1407/2827] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:59:50 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:51.089735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c28ba-4e93-4330-b03d-13e91e7d0a92 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:51.100964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c28ba-4e93-4330-b03d-13e91e7d0a92 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:51.155996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f967d-f1ad-4288-a206-77ab86a65587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:51.156703 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f967d-f1ad-4288-a206-77ab86a65587 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:51.156988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f967d-f1ad-4288-a206-77ab86a65587 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:51.157261 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f967d-f1ad-4288-a206-77ab86a65587 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:51.157966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f967d-f1ad-4288-a206-77ab86a65587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:51.160451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f967d-f1ad-4288-a206-77ab86a65587 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:51.177091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:51.183876 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1421/2828] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:59:51 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:51.187949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a12636-fd36-41b8-8b13-109b21a4bda3 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:51.198688 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a12636-fd36-41b8-8b13-109b21a4bda3 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:51.203555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4af8a-165c-4261-ba41-be56de7d7377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:51.207336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4af8a-165c-4261-ba41-be56de7d7377 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:51.207336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4af8a-165c-4261-ba41-be56de7d7377 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:51.207336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4af8a-165c-4261-ba41-be56de7d7377 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:51.207336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4af8a-165c-4261-ba41-be56de7d7377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:51.207770 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4af8a-165c-4261-ba41-be56de7d7377 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:51.219538 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1408/2829] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:51.224536 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f967d-f1ad-4288-a206-77ab86a65587 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:51.234798 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f967d-f1ad-4288-a206-77ab86a65587 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:51.239804 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819d43b-cb15-4278-8991-8eb79ae193ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:51.240944 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819d43b-cb15-4278-8991-8eb79ae193ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:51.241239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819d43b-cb15-4278-8991-8eb79ae193ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:51.241523 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819d43b-cb15-4278-8991-8eb79ae193ac admin admin] PATH_INFO: `/v3/projects/9340fc1a90e24c17b4d391e1c880e9b9/users/da808a44483b40ff84b58ad52bcd58ca/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:51.259753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819d43b-cb15-4278-8991-8eb79ae193ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=9340fc1a90e24c17b4d391e1c880e9b9, user_id=da808a44483b40ff84b58ad52bcd58ca, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:51.262684 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819d43b-cb15-4278-8991-8eb79ae193ac admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:51.276475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1422/2830] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:51 2022] PUT /identity/v3/projects/9340fc1a90e24c17b4d391e1c880e9b9/users/da808a44483b40ff84b58ad52bcd58ca/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:51.281028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4af8a-165c-4261-ba41-be56de7d7377 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:51.292140 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4af8a-165c-4261-ba41-be56de7d7377 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:51.354017 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199029f-6f38-4653-b036-21b9e3faf0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:51.354605 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199029f-6f38-4653-b036-21b9e3faf0df admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:51.354918 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199029f-6f38-4653-b036-21b9e3faf0df admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:51.355192 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199029f-6f38-4653-b036-21b9e3faf0df admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:51.355922 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199029f-6f38-4653-b036-21b9e3faf0df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:51.357542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199029f-6f38-4653-b036-21b9e3faf0df admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:51.367666 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1409/2831] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:51.372174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819d43b-cb15-4278-8991-8eb79ae193ac admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:51.385682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819d43b-cb15-4278-8991-8eb79ae193ac admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:51.390342 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a389c80-c849-43d3-b901-5b860dba19b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:51.391223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a389c80-c849-43d3-b901-5b860dba19b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:51.391512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a389c80-c849-43d3-b901-5b860dba19b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:51.391788 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a389c80-c849-43d3-b901-5b860dba19b5 admin admin] PATH_INFO: `/v3/projects/9340fc1a90e24c17b4d391e1c880e9b9/users/da808a44483b40ff84b58ad52bcd58ca/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:51.394388 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a389c80-c849-43d3-b901-5b860dba19b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9340fc1a90e24c17b4d391e1c880e9b9, user_id=da808a44483b40ff84b58ad52bcd58ca, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:51.397052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a389c80-c849-43d3-b901-5b860dba19b5 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:51.407945 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1423/2832] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:51 2022] PUT /identity/v3/projects/9340fc1a90e24c17b4d391e1c880e9b9/users/da808a44483b40ff84b58ad52bcd58ca/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:51.419829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33dd60-5911-4eff-9d48-54e99e3aee8d None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:51.419829 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33dd60-5911-4eff-9d48-54e99e3aee8d None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:51.420122 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33dd60-5911-4eff-9d48-54e99e3aee8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:51.441598 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:51.449990 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-6c33dd60-5911-4eff-9d48-54e99e3aee8d None None] MFA Rules not processed for user `da808a44483b40ff84b58ad52bcd58ca`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:51.451555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-6c33dd60-5911-4eff-9d48-54e99e3aee8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:51.529917 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1410/2833] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:51.541918 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a389c80-c849-43d3-b901-5b860dba19b5 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:51.543735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3a389c80-c849-43d3-b901-5b860dba19b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:51.603650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a389c80-c849-43d3-b901-5b860dba19b5 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:51.607566 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c792f16-ae8c-4803-bb92-4977affa2cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:51.608215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c792f16-ae8c-4803-bb92-4977affa2cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:51.608503 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c792f16-ae8c-4803-bb92-4977affa2cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:51.608811 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c792f16-ae8c-4803-bb92-4977affa2cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:51.618495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c792f16-ae8c-4803-bb92-4977affa2cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:51.619976 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c792f16-ae8c-4803-bb92-4977affa2cf0 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:51.630650 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1424/2834] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:51.737609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33dd60-5911-4eff-9d48-54e99e3aee8d None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:51.749018 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33dd60-5911-4eff-9d48-54e99e3aee8d None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:51.754084 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3958b2e-cd0a-4e6d-825d-492200145ff0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:51.754780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3958b2e-cd0a-4e6d-825d-492200145ff0 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:51.755066 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3958b2e-cd0a-4e6d-825d-492200145ff0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:51.755344 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3958b2e-cd0a-4e6d-825d-492200145ff0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:51.817336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3958b2e-cd0a-4e6d-825d-492200145ff0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:51.818855 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3958b2e-cd0a-4e6d-825d-492200145ff0 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:51.828745 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1411/2835] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:54.481349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c792f16-ae8c-4803-bb92-4977affa2cf0 service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:54.575297 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c792f16-ae8c-4803-bb92-4977affa2cf0 service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:54.580227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:54.581725 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:54.582005 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:54.582571 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] PATH_INFO: `/v3/users/ce7d85a5a9474e3bbb999e9ae64df152` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:54.583512 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce7d85a5a9474e3bbb999e9ae64df152)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:54.585468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:54.642968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce7d85a5a9474e3bbb999e9ae64df152'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:54.644448 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce7d85a5a9474e3bbb999e9ae64df152'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:54.653232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce7d85a5a9474e3bbb999e9ae64df152'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:54.662475 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:54.668822 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce7d85a5a9474e3bbb999e9ae64df152'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:54.679079 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce7d85a5a9474e3bbb999e9ae64df152'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:54.685886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1425/2836] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:59:54 2022] DELETE /identity/v3/users/ce7d85a5a9474e3bbb999e9ae64df152 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:54.691173 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02020bb-e10a-46d5-b773-c3d7e9b2856a None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:54.691667 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02020bb-e10a-46d5-b773-c3d7e9b2856a None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:54.691932 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02020bb-e10a-46d5-b773-c3d7e9b2856a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:54.712479 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-a02020bb-e10a-46d5-b773-c3d7e9b2856a None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:54.714138 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a02020bb-e10a-46d5-b773-c3d7e9b2856a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:54.783263 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1412/2837] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:54.793172 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:54.795653 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:54.864246 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f9972-133b-4554-bba2-f480eafc0d8f admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:54.867925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61c30f-7411-4394-bb96-af4febd4c9bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:54.868648 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61c30f-7411-4394-bb96-af4febd4c9bf service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:54.868957 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61c30f-7411-4394-bb96-af4febd4c9bf service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:54.869299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61c30f-7411-4394-bb96-af4febd4c9bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:54.881974 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61c30f-7411-4394-bb96-af4febd4c9bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:54.883477 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61c30f-7411-4394-bb96-af4febd4c9bf service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:54.896410 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1426/2838] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:55.201408 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02020bb-e10a-46d5-b773-c3d7e9b2856a None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:55.202952 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a02020bb-e10a-46d5-b773-c3d7e9b2856a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:55.224088 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02020bb-e10a-46d5-b773-c3d7e9b2856a None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:55.229865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:55.230824 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:55.231109 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:55.231391 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] PATH_INFO: `/v3/projects/bdf850fce06f4d558da0b4bee7dfe728` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:55.232675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf850fce06f4d558da0b4bee7dfe728)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:55.234609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:55.271661 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:55.288210 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf850fce06f4d558da0b4bee7dfe728', 'request_id': 'req-7ee15c06-8243-4112-a84f-b3de63fda190'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.296768 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf850fce06f4d558da0b4bee7dfe728', 'request_id': 'req-7ee15c06-8243-4112-a84f-b3de63fda190'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.303367 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf850fce06f4d558da0b4bee7dfe728', 'request_id': 'req-7ee15c06-8243-4112-a84f-b3de63fda190'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.315609 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] The token cache is being invalidate because project bdf850fce06f4d558da0b4bee7dfe728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:59:55.315900 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.316596 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.318145 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1413/2839] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:59:55 2022] DELETE /identity/v3/projects/bdf850fce06f4d558da0b4bee7dfe728 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:55.322529 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61c30f-7411-4394-bb96-af4febd4c9bf service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:55.332253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61c30f-7411-4394-bb96-af4febd4c9bf service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:55.391447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63912a1-6885-4d01-964b-1ef1b3c2bac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:55.392334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63912a1-6885-4d01-964b-1ef1b3c2bac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:55.392645 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63912a1-6885-4d01-964b-1ef1b3c2bac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:55.392925 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63912a1-6885-4d01-964b-1ef1b3c2bac3 admin admin] PATH_INFO: `/v3/users/da808a44483b40ff84b58ad52bcd58ca` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:55.394199 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63912a1-6885-4d01-964b-1ef1b3c2bac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da808a44483b40ff84b58ad52bcd58ca)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:55.396138 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63912a1-6885-4d01-964b-1ef1b3c2bac3 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:55.458644 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b63912a1-6885-4d01-964b-1ef1b3c2bac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da808a44483b40ff84b58ad52bcd58ca'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.459487 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b63912a1-6885-4d01-964b-1ef1b3c2bac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da808a44483b40ff84b58ad52bcd58ca'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.467568 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b63912a1-6885-4d01-964b-1ef1b3c2bac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da808a44483b40ff84b58ad52bcd58ca'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.480613 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b63912a1-6885-4d01-964b-1ef1b3c2bac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da808a44483b40ff84b58ad52bcd58ca'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.487902 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b63912a1-6885-4d01-964b-1ef1b3c2bac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da808a44483b40ff84b58ad52bcd58ca'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.494175 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1427/2840] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:59:55 2022] DELETE /identity/v3/users/da808a44483b40ff84b58ad52bcd58ca => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:59:55.794530 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:55.796004 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:55.811790 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee15c06-8243-4112-a84f-b3de63fda190 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:55.874038 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0b987-3360-467e-aee3-1d0eb16da8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:55.874928 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0b987-3360-467e-aee3-1d0eb16da8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:55.875215 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0b987-3360-467e-aee3-1d0eb16da8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:55.875490 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0b987-3360-467e-aee3-1d0eb16da8fb admin admin] PATH_INFO: `/v3/projects/9340fc1a90e24c17b4d391e1c880e9b9` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:55.876724 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0b987-3360-467e-aee3-1d0eb16da8fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9340fc1a90e24c17b4d391e1c880e9b9)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:55.879159 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0b987-3360-467e-aee3-1d0eb16da8fb admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:55.928715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2ba0b987-3360-467e-aee3-1d0eb16da8fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9340fc1a90e24c17b4d391e1c880e9b9', 'request_id': 'req-2ba0b987-3360-467e-aee3-1d0eb16da8fb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.936174 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2ba0b987-3360-467e-aee3-1d0eb16da8fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9340fc1a90e24c17b4d391e1c880e9b9', 'request_id': 'req-2ba0b987-3360-467e-aee3-1d0eb16da8fb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.942903 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2ba0b987-3360-467e-aee3-1d0eb16da8fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9340fc1a90e24c17b4d391e1c880e9b9', 'request_id': 'req-2ba0b987-3360-467e-aee3-1d0eb16da8fb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.954290 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2ba0b987-3360-467e-aee3-1d0eb16da8fb admin admin] The token cache is being invalidate because project 9340fc1a90e24c17b4d391e1c880e9b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:59:55.954581 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2ba0b987-3360-467e-aee3-1d0eb16da8fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.956809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-2ba0b987-3360-467e-aee3-1d0eb16da8fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:55.958503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1414/2841] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:59:55 2022] DELETE /identity/v3/projects/9340fc1a90e24c17b4d391e1c880e9b9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:55.963385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d67321-4621-4fdf-8a50-f52bf5e6862e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:55.963674 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d67321-4621-4fdf-8a50-f52bf5e6862e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:55.963955 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d67321-4621-4fdf-8a50-f52bf5e6862e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:55.972892 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:55.985054 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-14d67321-4621-4fdf-8a50-f52bf5e6862e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:55.986646 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-14d67321-4621-4fdf-8a50-f52bf5e6862e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:56.048332 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1428/2842] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:56.056092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc40343-26cb-4220-86fe-41f1cdbb56d9 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:56.056379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc40343-26cb-4220-86fe-41f1cdbb56d9 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:56.056765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc40343-26cb-4220-86fe-41f1cdbb56d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:56.077216 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-9cc40343-26cb-4220-86fe-41f1cdbb56d9 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:56.078451 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-9cc40343-26cb-4220-86fe-41f1cdbb56d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:56.087155 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1415/2843] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:56.091612 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d67321-4621-4fdf-8a50-f52bf5e6862e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:56.101787 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d67321-4621-4fdf-8a50-f52bf5e6862e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:56.106620 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b6276-aee7-4817-9ef3-f48af50a31fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:56.107289 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b6276-aee7-4817-9ef3-f48af50a31fa admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:56.107576 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b6276-aee7-4817-9ef3-f48af50a31fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:56.107854 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b6276-aee7-4817-9ef3-f48af50a31fa admin admin] PATH_INFO: `/v3/domains` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:56.108611 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b6276-aee7-4817-9ef3-f48af50a31fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86752) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 08:59:56.108927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b6276-aee7-4817-9ef3-f48af50a31fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:56.110538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b6276-aee7-4817-9ef3-f48af50a31fa admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:56.123110 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1429/2844] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 08:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 08:59:56.127476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc40343-26cb-4220-86fe-41f1cdbb56d9 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:56.138634 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc40343-26cb-4220-86fe-41f1cdbb56d9 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:56.144350 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99496d-7a4f-4d26-b44d-abe09cdcffe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:56.145064 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99496d-7a4f-4d26-b44d-abe09cdcffe6 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:56.145365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99496d-7a4f-4d26-b44d-abe09cdcffe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:56.145643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99496d-7a4f-4d26-b44d-abe09cdcffe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:56.146369 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99496d-7a4f-4d26-b44d-abe09cdcffe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:56.148290 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99496d-7a4f-4d26-b44d-abe09cdcffe6 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:56.178146 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1416/2845] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:59:56 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:56.182662 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b6276-aee7-4817-9ef3-f48af50a31fa admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:56.194882 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b6276-aee7-4817-9ef3-f48af50a31fa admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:56.257702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645876f-1e48-405a-8311-e2a2e03b8d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:56.258390 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645876f-1e48-405a-8311-e2a2e03b8d97 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:56.258916 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645876f-1e48-405a-8311-e2a2e03b8d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:56.259210 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645876f-1e48-405a-8311-e2a2e03b8d97 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:56.259968 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645876f-1e48-405a-8311-e2a2e03b8d97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:56.262947 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645876f-1e48-405a-8311-e2a2e03b8d97 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:56.289712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1430/2846] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:59:56 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:56.298619 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99496d-7a4f-4d26-b44d-abe09cdcffe6 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:56.311309 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99496d-7a4f-4d26-b44d-abe09cdcffe6 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:56.317671 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d302d9-14a3-44db-830f-5ba7ec525d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:56.318436 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d302d9-14a3-44db-830f-5ba7ec525d64 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:56.318793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d302d9-14a3-44db-830f-5ba7ec525d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:56.319142 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d302d9-14a3-44db-830f-5ba7ec525d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:56.319966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d302d9-14a3-44db-830f-5ba7ec525d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:56.321686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d302d9-14a3-44db-830f-5ba7ec525d64 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:56.333417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1417/2847] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:56.337706 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645876f-1e48-405a-8311-e2a2e03b8d97 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:56.347809 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645876f-1e48-405a-8311-e2a2e03b8d97 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:56.352540 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c95384-5eae-4b5f-828d-e449e6ca95e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:56.353263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c95384-5eae-4b5f-828d-e449e6ca95e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:56.353546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c95384-5eae-4b5f-828d-e449e6ca95e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:56.353827 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c95384-5eae-4b5f-828d-e449e6ca95e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:56.354617 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c95384-5eae-4b5f-828d-e449e6ca95e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:56.356215 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c95384-5eae-4b5f-828d-e449e6ca95e3 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:56.367882 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1431/2848] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:56.372366 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d302d9-14a3-44db-830f-5ba7ec525d64 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:56.383319 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d302d9-14a3-44db-830f-5ba7ec525d64 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:56.388204 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7d3e8-076b-43b3-8047-96d46fa95cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:56.389133 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7d3e8-076b-43b3-8047-96d46fa95cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:56.389417 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7d3e8-076b-43b3-8047-96d46fa95cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:56.389695 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7d3e8-076b-43b3-8047-96d46fa95cd9 admin admin] PATH_INFO: `/v3/projects/43fd8da6f63148078de9f47e8ae1bbfa/users/c49a0d92430e4f20aa741b6f0e5b94d4/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:56.393073 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:56.406252 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7d3e8-076b-43b3-8047-96d46fa95cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43fd8da6f63148078de9f47e8ae1bbfa, user_id=c49a0d92430e4f20aa741b6f0e5b94d4, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:56.409588 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7d3e8-076b-43b3-8047-96d46fa95cd9 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:56.422488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1418/2849] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:56 2022] PUT /identity/v3/projects/43fd8da6f63148078de9f47e8ae1bbfa/users/c49a0d92430e4f20aa741b6f0e5b94d4/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:56.490295 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19028caf-1c53-43af-aecb-a4bca72d8e62 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:56.490855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19028caf-1c53-43af-aecb-a4bca72d8e62 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:56.490855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19028caf-1c53-43af-aecb-a4bca72d8e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:56.520958 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-19028caf-1c53-43af-aecb-a4bca72d8e62 None None] MFA Rules not processed for user `c49a0d92430e4f20aa741b6f0e5b94d4`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:56.522227 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-19028caf-1c53-43af-aecb-a4bca72d8e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:56.587051 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:56.592467 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1432/2850] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:56.595611 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7d3e8-076b-43b3-8047-96d46fa95cd9 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:56.661135 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7d3e8-076b-43b3-8047-96d46fa95cd9 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:56.666763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684bf88-239a-4435-923b-5be8e6bdd61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:56.666763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684bf88-239a-4435-923b-5be8e6bdd61f admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:56.666763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684bf88-239a-4435-923b-5be8e6bdd61f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:56.667345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684bf88-239a-4435-923b-5be8e6bdd61f admin admin] PATH_INFO: `/v3/projects` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:56.667665 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684bf88-239a-4435-923b-5be8e6bdd61f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:56.669593 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684bf88-239a-4435-923b-5be8e6bdd61f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:56.702896 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1419/2851] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 08:59:56 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:56.704291 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19028caf-1c53-43af-aecb-a4bca72d8e62 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:56.717425 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19028caf-1c53-43af-aecb-a4bca72d8e62 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:56.784835 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab4e3a-7d6a-4541-9297-60b5d55aff13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:56.785540 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab4e3a-7d6a-4541-9297-60b5d55aff13 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:56.786017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab4e3a-7d6a-4541-9297-60b5d55aff13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:56.786334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab4e3a-7d6a-4541-9297-60b5d55aff13 admin admin] PATH_INFO: `/v3/users` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:56.787048 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab4e3a-7d6a-4541-9297-60b5d55aff13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:56.789031 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab4e3a-7d6a-4541-9297-60b5d55aff13 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:56.815019 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1433/2852] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 08:59:56 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 08:59:56.816316 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684bf88-239a-4435-923b-5be8e6bdd61f admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:56.827520 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684bf88-239a-4435-923b-5be8e6bdd61f admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:56.832363 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e9841-fcc8-4250-ae72-f0f4d71c9c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:56.833076 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e9841-fcc8-4250-ae72-f0f4d71c9c7e admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:56.833359 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e9841-fcc8-4250-ae72-f0f4d71c9c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:56.833639 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e9841-fcc8-4250-ae72-f0f4d71c9c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:56.834400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e9841-fcc8-4250-ae72-f0f4d71c9c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:56.836022 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e9841-fcc8-4250-ae72-f0f4d71c9c7e admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:56.850169 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1420/2853] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:56.854284 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab4e3a-7d6a-4541-9297-60b5d55aff13 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:56.866746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab4e3a-7d6a-4541-9297-60b5d55aff13 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:56.871970 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf5eb3-fb2b-485c-99cb-850c79009d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:56.873182 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf5eb3-fb2b-485c-99cb-850c79009d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:56.873430 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf5eb3-fb2b-485c-99cb-850c79009d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:56.873711 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf5eb3-fb2b-485c-99cb-850c79009d70 admin admin] PATH_INFO: `/v3/projects/d3191d52c0474daa812b9e6916f05327/users/5a5079c822904033b172976697265b26/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:56.896924 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf5eb3-fb2b-485c-99cb-850c79009d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3191d52c0474daa812b9e6916f05327, user_id=5a5079c822904033b172976697265b26, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:56.900505 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf5eb3-fb2b-485c-99cb-850c79009d70 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:56.915561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1434/2854] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:56 2022] PUT /identity/v3/projects/d3191d52c0474daa812b9e6916f05327/users/5a5079c822904033b172976697265b26/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:59:56.920517 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e9841-fcc8-4250-ae72-f0f4d71c9c7e admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:56.933467 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e9841-fcc8-4250-ae72-f0f4d71c9c7e admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:57.000476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a2f9b-3a17-410c-a902-58227d239d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:57.001220 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a2f9b-3a17-410c-a902-58227d239d81 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:57.001504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a2f9b-3a17-410c-a902-58227d239d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:57.001780 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a2f9b-3a17-410c-a902-58227d239d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:57.002526 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a2f9b-3a17-410c-a902-58227d239d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:57.004136 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a2f9b-3a17-410c-a902-58227d239d81 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:57.015624 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1421/2855] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 08:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 08:59:57.020335 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf5eb3-fb2b-485c-99cb-850c79009d70 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:57.031591 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf5eb3-fb2b-485c-99cb-850c79009d70 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:57.037135 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14f0aa-079d-4808-999e-fa28685b45ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:57.038084 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14f0aa-079d-4808-999e-fa28685b45ba admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:57.038370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14f0aa-079d-4808-999e-fa28685b45ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:57.038647 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14f0aa-079d-4808-999e-fa28685b45ba admin admin] PATH_INFO: `/v3/projects/d3191d52c0474daa812b9e6916f05327/users/5a5079c822904033b172976697265b26/roles/996428c7fd054e5cb6a1aaa31f27cf84` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:57.040777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14f0aa-079d-4808-999e-fa28685b45ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3191d52c0474daa812b9e6916f05327, user_id=5a5079c822904033b172976697265b26, role_id=996428c7fd054e5cb6a1aaa31f27cf84)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:57.043458 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14f0aa-079d-4808-999e-fa28685b45ba admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:57.054805 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1435/2856] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 08:59:57 2022] PUT /identity/v3/projects/d3191d52c0474daa812b9e6916f05327/users/5a5079c822904033b172976697265b26/roles/996428c7fd054e5cb6a1aaa31f27cf84 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 08:59:57.064787 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ad72f-9512-46f8-a15b-959357278ffe None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:57.065083 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ad72f-9512-46f8-a15b-959357278ffe None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:57.065355 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ad72f-9512-46f8-a15b-959357278ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:57.095299 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-4f4ad72f-9512-46f8-a15b-959357278ffe None None] MFA Rules not processed for user `5a5079c822904033b172976697265b26`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:57.097182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4f4ad72f-9512-46f8-a15b-959357278ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:57.146168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14f0aa-079d-4808-999e-fa28685b45ba admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:57.146981 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:57.147886 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-5c14f0aa-079d-4808-999e-fa28685b45ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:57.175556 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1422/2857] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:57.186264 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ad72f-9512-46f8-a15b-959357278ffe None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:57.188032 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-4f4ad72f-9512-46f8-a15b-959357278ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:57.211851 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14f0aa-079d-4808-999e-fa28685b45ba admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:57.217965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e14afd-7185-4720-8dc2-c7e4643d0bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:57.218637 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e14afd-7185-4720-8dc2-c7e4643d0bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:57.218923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e14afd-7185-4720-8dc2-c7e4643d0bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:57.219201 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e14afd-7185-4720-8dc2-c7e4643d0bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:57.221113 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-01e14afd-7185-4720-8dc2-c7e4643d0bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:57.252986 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ad72f-9512-46f8-a15b-959357278ffe None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:57.256860 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401e513-d86f-4ba2-8ba2-6ed5a43636d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:57.257528 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401e513-d86f-4ba2-8ba2-6ed5a43636d6 service cinder] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:57.257811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401e513-d86f-4ba2-8ba2-6ed5a43636d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:57.258089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401e513-d86f-4ba2-8ba2-6ed5a43636d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:57.267809 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401e513-d86f-4ba2-8ba2-6ed5a43636d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:57.269822 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401e513-d86f-4ba2-8ba2-6ed5a43636d6 service cinder] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:57.280435 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1423/2858] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:59:57.295801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e14afd-7185-4720-8dc2-c7e4643d0bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:57.297321 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e14afd-7185-4720-8dc2-c7e4643d0bd5 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:57.311577 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1436/2859] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 08:59:57.892725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401e513-d86f-4ba2-8ba2-6ed5a43636d6 service cinder] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:57.958456 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401e513-d86f-4ba2-8ba2-6ed5a43636d6 service cinder] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:57.963119 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:57.964040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:57.964329 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:57.964635 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] PATH_INFO: `/v3/users/c49a0d92430e4f20aa741b6f0e5b94d4` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:57.965879 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c49a0d92430e4f20aa741b6f0e5b94d4)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:57.967845 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:58.018395 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c49a0d92430e4f20aa741b6f0e5b94d4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:58.026818 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c49a0d92430e4f20aa741b6f0e5b94d4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:58.041131 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c49a0d92430e4f20aa741b6f0e5b94d4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:58.047891 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c49a0d92430e4f20aa741b6f0e5b94d4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:58.048564 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c49a0d92430e4f20aa741b6f0e5b94d4'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:58.055234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1424/2860] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:59:57 2022] DELETE /identity/v3/users/c49a0d92430e4f20aa741b6f0e5b94d4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:58.060126 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1078a-de68-42f4-9011-2b03283d781e None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:58.060688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1078a-de68-42f4-9011-2b03283d781e None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:58.060688 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1078a-de68-42f4-9011-2b03283d781e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:58.068546 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:58.084538 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-cdf1078a-de68-42f4-9011-2b03283d781e None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:58.085900 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-cdf1078a-de68-42f4-9011-2b03283d781e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:58.149377 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1437/2861] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:58.159673 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:58.161293 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:58.166889 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:58.234476 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e307a-4607-4e25-9151-67a5d5f79beb admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:58.238754 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b6c28-027c-48a6-a7eb-9f97283d2d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:58.239414 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b6c28-027c-48a6-a7eb-9f97283d2d2f service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:58.239698 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b6c28-027c-48a6-a7eb-9f97283d2d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:58.239977 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b6c28-027c-48a6-a7eb-9f97283d2d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:58.250007 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b6c28-027c-48a6-a7eb-9f97283d2d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:58.251605 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b6c28-027c-48a6-a7eb-9f97283d2d2f service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:58.261843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1425/2862] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 08:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 08:59:59.027279 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1078a-de68-42f4-9011-2b03283d781e None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:59.029746 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-cdf1078a-de68-42f4-9011-2b03283d781e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:59.042221 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1078a-de68-42f4-9011-2b03283d781e None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:59.047421 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:59.048425 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:59.048806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:59.049174 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] PATH_INFO: `/v3/projects/43fd8da6f63148078de9f47e8ae1bbfa` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:59.050541 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43fd8da6f63148078de9f47e8ae1bbfa)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:59.052561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:59.111965 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43fd8da6f63148078de9f47e8ae1bbfa', 'request_id': 'req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.122166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43fd8da6f63148078de9f47e8ae1bbfa', 'request_id': 'req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.129462 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43fd8da6f63148078de9f47e8ae1bbfa', 'request_id': 'req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.142196 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] The token cache is being invalidate because project 43fd8da6f63148078de9f47e8ae1bbfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:59:59.142539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.143114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.145367 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1438/2863] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:59:59 2022] DELETE /identity/v3/projects/43fd8da6f63148078de9f47e8ae1bbfa => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:59.149356 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b6c28-027c-48a6-a7eb-9f97283d2d2f service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:59.160418 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b6c28-027c-48a6-a7eb-9f97283d2d2f service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:59.226113 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c508806-19b1-4aa3-b58d-b1d8526b9c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:59.227006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c508806-19b1-4aa3-b58d-b1d8526b9c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:59.227288 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c508806-19b1-4aa3-b58d-b1d8526b9c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:59.227559 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c508806-19b1-4aa3-b58d-b1d8526b9c5f admin admin] PATH_INFO: `/v3/users/5a5079c822904033b172976697265b26` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:59.228984 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c508806-19b1-4aa3-b58d-b1d8526b9c5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a5079c822904033b172976697265b26)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:59.231231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c508806-19b1-4aa3-b58d-b1d8526b9c5f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:59.286400 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0c508806-19b1-4aa3-b58d-b1d8526b9c5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a5079c822904033b172976697265b26'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.298186 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0c508806-19b1-4aa3-b58d-b1d8526b9c5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a5079c822904033b172976697265b26'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.312510 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0c508806-19b1-4aa3-b58d-b1d8526b9c5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a5079c822904033b172976697265b26'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.321320 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0c508806-19b1-4aa3-b58d-b1d8526b9c5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a5079c822904033b172976697265b26'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.323018 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0c508806-19b1-4aa3-b58d-b1d8526b9c5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a5079c822904033b172976697265b26'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.330087 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1426/2864] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 08:59:59 2022] DELETE /identity/v3/users/5a5079c822904033b172976697265b26 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:59.650845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 08:59:59.652980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:59.671885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93f63e-21a9-4cdb-93fc-0642fae0a4c0 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 08:59:59.761899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358d127-5b23-4bc0-9f80-f40e7695e40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 08:59:59.762897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358d127-5b23-4bc0-9f80-f40e7695e40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:59.763190 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358d127-5b23-4bc0-9f80-f40e7695e40b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:59.763551 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358d127-5b23-4bc0-9f80-f40e7695e40b admin admin] PATH_INFO: `/v3/projects/d3191d52c0474daa812b9e6916f05327` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:59.765300 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358d127-5b23-4bc0-9f80-f40e7695e40b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3191d52c0474daa812b9e6916f05327)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 08:59:59.767036 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358d127-5b23-4bc0-9f80-f40e7695e40b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 08:59:59.807624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:59.830749 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c358d127-5b23-4bc0-9f80-f40e7695e40b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3191d52c0474daa812b9e6916f05327', 'request_id': 'req-c358d127-5b23-4bc0-9f80-f40e7695e40b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.839186 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c358d127-5b23-4bc0-9f80-f40e7695e40b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3191d52c0474daa812b9e6916f05327', 'request_id': 'req-c358d127-5b23-4bc0-9f80-f40e7695e40b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.848504 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c358d127-5b23-4bc0-9f80-f40e7695e40b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3191d52c0474daa812b9e6916f05327', 'request_id': 'req-c358d127-5b23-4bc0-9f80-f40e7695e40b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.864259 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c358d127-5b23-4bc0-9f80-f40e7695e40b admin admin] The token cache is being invalidate because project d3191d52c0474daa812b9e6916f05327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 08:59:59.864663 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c358d127-5b23-4bc0-9f80-f40e7695e40b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.865440 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-c358d127-5b23-4bc0-9f80-f40e7695e40b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 08:59:59.867318 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1439/2865] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 08:59:59 2022] DELETE /identity/v3/projects/d3191d52c0474daa812b9e6916f05327 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 08:59:59.873385 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48861dc6-26c5-466c-951a-594259c4c3d8 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:59.873730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48861dc6-26c5-466c-951a-594259c4c3d8 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:59.873963 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48861dc6-26c5-466c-951a-594259c4c3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 08:59:59.895011 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-48861dc6-26c5-466c-951a-594259c4c3d8 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 08:59:59.896250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-48861dc6-26c5-466c-951a-594259c4c3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 08:59:59.917960 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 08:59:59.977652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1427/2866] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 08:59:59.986032 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea3688-7370-46f9-a63d-4eddc02a6f2f None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 08:59:59.986438 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea3688-7370-46f9-a63d-4eddc02a6f2f None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 08:59:59.987081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea3688-7370-46f9-a63d-4eddc02a6f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:00.011045 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-bcea3688-7370-46f9-a63d-4eddc02a6f2f None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 09:00:00.012296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-bcea3688-7370-46f9-a63d-4eddc02a6f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:00.022466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1440/2867] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 08:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 09:00:00.027089 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48861dc6-26c5-466c-951a-594259c4c3d8 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:00.038820 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48861dc6-26c5-466c-951a-594259c4c3d8 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:00.044190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c69456-4db5-4e6e-a6f7-3fa3361de16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:00.044914 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c69456-4db5-4e6e-a6f7-3fa3361de16f admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:00.045201 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c69456-4db5-4e6e-a6f7-3fa3361de16f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:00.045480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c69456-4db5-4e6e-a6f7-3fa3361de16f admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:00.046231 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c69456-4db5-4e6e-a6f7-3fa3361de16f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 09:00:00.046542 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c69456-4db5-4e6e-a6f7-3fa3361de16f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:00.048161 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c69456-4db5-4e6e-a6f7-3fa3361de16f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:00.061870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1428/2868] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 09:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 09:00:00.066465 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea3688-7370-46f9-a63d-4eddc02a6f2f None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:00.076132 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea3688-7370-46f9-a63d-4eddc02a6f2f None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:00.081479 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362b61d-ca1f-4198-9a9d-a8ab4feee555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:00.082160 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362b61d-ca1f-4198-9a9d-a8ab4feee555 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:00.082447 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362b61d-ca1f-4198-9a9d-a8ab4feee555 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:00.082732 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362b61d-ca1f-4198-9a9d-a8ab4feee555 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:00.083452 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362b61d-ca1f-4198-9a9d-a8ab4feee555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:00.085526 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362b61d-ca1f-4198-9a9d-a8ab4feee555 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:00.123451 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1441/2869] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 09:00:00 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 09:00:00.127816 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c69456-4db5-4e6e-a6f7-3fa3361de16f admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:00.139740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c69456-4db5-4e6e-a6f7-3fa3361de16f admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:00.211666 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ae5c6-aa10-483c-8b6c-67cf9c37de20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:00.212560 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ae5c6-aa10-483c-8b6c-67cf9c37de20 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:00.212793 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ae5c6-aa10-483c-8b6c-67cf9c37de20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:00.213078 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ae5c6-aa10-483c-8b6c-67cf9c37de20 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:00.214047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ae5c6-aa10-483c-8b6c-67cf9c37de20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:00.216025 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ae5c6-aa10-483c-8b6c-67cf9c37de20 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:00.243028 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1429/2870] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 09:00:00 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 09:00:00.244362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362b61d-ca1f-4198-9a9d-a8ab4feee555 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:00.255635 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362b61d-ca1f-4198-9a9d-a8ab4feee555 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:00.261017 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0176aa-0d03-497b-a3f8-ed85a109b52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:00.261707 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0176aa-0d03-497b-a3f8-ed85a109b52b admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:00.261996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0176aa-0d03-497b-a3f8-ed85a109b52b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:00.262276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0176aa-0d03-497b-a3f8-ed85a109b52b admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:00.263016 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0176aa-0d03-497b-a3f8-ed85a109b52b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:00.264666 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0176aa-0d03-497b-a3f8-ed85a109b52b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:00.276808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1442/2871] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 09:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 09:00:00.281267 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ae5c6-aa10-483c-8b6c-67cf9c37de20 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:00.292312 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ae5c6-aa10-483c-8b6c-67cf9c37de20 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:00.297321 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b81b2c-d1c9-47e6-80dc-97fdf724e059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:00.298012 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b81b2c-d1c9-47e6-80dc-97fdf724e059 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:00.298295 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b81b2c-d1c9-47e6-80dc-97fdf724e059 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:00.298572 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b81b2c-d1c9-47e6-80dc-97fdf724e059 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:00.299324 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b81b2c-d1c9-47e6-80dc-97fdf724e059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:00.305100 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b81b2c-d1c9-47e6-80dc-97fdf724e059 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:00.313256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1430/2872] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 09:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 09:00:00.317670 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0176aa-0d03-497b-a3f8-ed85a109b52b admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:00.328905 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0176aa-0d03-497b-a3f8-ed85a109b52b admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:00.337094 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185ce8a-55db-4631-be42-15858b14f743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:00.338091 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185ce8a-55db-4631-be42-15858b14f743 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:00.338453 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185ce8a-55db-4631-be42-15858b14f743 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:00.338728 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185ce8a-55db-4631-be42-15858b14f743 admin admin] PATH_INFO: `/v3/projects/8a5b18b5ce9f4150a907ff8b7ad35b8d/users/3a3c5071c88c4a88b9cf4a6ba82cc3a0/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:00.357168 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185ce8a-55db-4631-be42-15858b14f743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5b18b5ce9f4150a907ff8b7ad35b8d, user_id=3a3c5071c88c4a88b9cf4a6ba82cc3a0, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:00.360037 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185ce8a-55db-4631-be42-15858b14f743 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:00.374283 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1443/2873] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 09:00:00 2022] PUT /identity/v3/projects/8a5b18b5ce9f4150a907ff8b7ad35b8d/users/3a3c5071c88c4a88b9cf4a6ba82cc3a0/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:00:00.386347 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab42c44-c79a-4779-a227-7eeabc80ca89 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:00.386643 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab42c44-c79a-4779-a227-7eeabc80ca89 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:00.386923 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab42c44-c79a-4779-a227-7eeabc80ca89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:00.417336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-8ab42c44-c79a-4779-a227-7eeabc80ca89 None None] MFA Rules not processed for user `3a3c5071c88c4a88b9cf4a6ba82cc3a0`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 09:00:00.418557 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8ab42c44-c79a-4779-a227-7eeabc80ca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:00.490867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1431/2874] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 09:00:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 09:00:00.503058 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185ce8a-55db-4631-be42-15858b14f743 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:00.504904 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-e185ce8a-55db-4631-be42-15858b14f743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:00.572799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185ce8a-55db-4631-be42-15858b14f743 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:00.578103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe848cb-26bc-4901-a607-05f11e182dd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:00.578912 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe848cb-26bc-4901-a607-05f11e182dd3 service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:00.579247 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe848cb-26bc-4901-a607-05f11e182dd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:00.579567 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe848cb-26bc-4901-a607-05f11e182dd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:00.590370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe848cb-26bc-4901-a607-05f11e182dd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:00.591865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe848cb-26bc-4901-a607-05f11e182dd3 service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:00.604845 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1444/2875] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:00:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:00:03.174690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab42c44-c79a-4779-a227-7eeabc80ca89 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:03.198578 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:00:03.243966 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab42c44-c79a-4779-a227-7eeabc80ca89 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:03.249081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:03.249962 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:03.250240 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:03.250518 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] PATH_INFO: `/v3/users/3a3c5071c88c4a88b9cf4a6ba82cc3a0` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:03.251725 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3c5071c88c4a88b9cf4a6ba82cc3a0)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:03.253727 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:03.303025 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3c5071c88c4a88b9cf4a6ba82cc3a0'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:03.311482 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3c5071c88c4a88b9cf4a6ba82cc3a0'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:03.324275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3c5071c88c4a88b9cf4a6ba82cc3a0'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:03.332868 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3c5071c88c4a88b9cf4a6ba82cc3a0'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:03.333544 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3c5071c88c4a88b9cf4a6ba82cc3a0'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:03.339812 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1432/2876] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 09:00:03 2022] DELETE /identity/v3/users/3a3c5071c88c4a88b9cf4a6ba82cc3a0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:00:03.344359 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fcad0-5932-4b4c-ad2f-20bc117682b1 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:03.344889 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fcad0-5932-4b4c-ad2f-20bc117682b1 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:03.345026 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fcad0-5932-4b4c-ad2f-20bc117682b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:03.364627 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-2a1fcad0-5932-4b4c-ad2f-20bc117682b1 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 09:00:03.365855 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2a1fcad0-5932-4b4c-ad2f-20bc117682b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:03.421693 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:00:03.436010 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1445/2877] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 09:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 09:00:03.446308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:03.447765 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:03.507811 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c774556-74bc-4487-9717-609e5cd57215 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:03.512503 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee8fda-9273-4b0b-a726-ee19e7364641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:03.513568 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee8fda-9273-4b0b-a726-ee19e7364641 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:03.514024 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee8fda-9273-4b0b-a726-ee19e7364641 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:03.514420 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee8fda-9273-4b0b-a726-ee19e7364641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:03.525534 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee8fda-9273-4b0b-a726-ee19e7364641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:03.527001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee8fda-9273-4b0b-a726-ee19e7364641 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:03.538325 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1433/2878] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:00:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:00:03.821907 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fcad0-5932-4b4c-ad2f-20bc117682b1 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:03.823524 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2a1fcad0-5932-4b4c-ad2f-20bc117682b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:03.836239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fcad0-5932-4b4c-ad2f-20bc117682b1 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:03.841298 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941c2f8-c81f-4041-8347-223797fccf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:03.842176 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941c2f8-c81f-4041-8347-223797fccf4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:03.842458 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941c2f8-c81f-4041-8347-223797fccf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:03.842735 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941c2f8-c81f-4041-8347-223797fccf4b admin admin] PATH_INFO: `/v3/projects/8a5b18b5ce9f4150a907ff8b7ad35b8d` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:03.843988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941c2f8-c81f-4041-8347-223797fccf4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a5b18b5ce9f4150a907ff8b7ad35b8d)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:03.845988 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941c2f8-c81f-4041-8347-223797fccf4b admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:03.901373 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a941c2f8-c81f-4041-8347-223797fccf4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a5b18b5ce9f4150a907ff8b7ad35b8d', 'request_id': 'req-a941c2f8-c81f-4041-8347-223797fccf4b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:03.909166 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a941c2f8-c81f-4041-8347-223797fccf4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a5b18b5ce9f4150a907ff8b7ad35b8d', 'request_id': 'req-a941c2f8-c81f-4041-8347-223797fccf4b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:03.914966 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a941c2f8-c81f-4041-8347-223797fccf4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a5b18b5ce9f4150a907ff8b7ad35b8d', 'request_id': 'req-a941c2f8-c81f-4041-8347-223797fccf4b'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:03.927444 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a941c2f8-c81f-4041-8347-223797fccf4b admin admin] The token cache is being invalidate because project 8a5b18b5ce9f4150a907ff8b7ad35b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 09:00:03.927658 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a941c2f8-c81f-4041-8347-223797fccf4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:03.928296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-a941c2f8-c81f-4041-8347-223797fccf4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:03.929900 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1446/2879] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 09:00:03 2022] DELETE /identity/v3/projects/8a5b18b5ce9f4150a907ff8b7ad35b8d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:00:03.934938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff67f43-1fa4-4e60-a6ed-a9d6c8351d65 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:03.935290 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff67f43-1fa4-4e60-a6ed-a9d6c8351d65 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:03.935580 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff67f43-1fa4-4e60-a6ed-a9d6c8351d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:03.956255 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-8ff67f43-1fa4-4e60-a6ed-a9d6c8351d65 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 09:00:03.957525 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8ff67f43-1fa4-4e60-a6ed-a9d6c8351d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:04.003788 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:00:04.026763 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1434/2880] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 09:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 09:00:04.034865 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097a1a6-9673-420b-bd0c-2b2fca6312fc None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:04.035152 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097a1a6-9673-420b-bd0c-2b2fca6312fc None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:04.035434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097a1a6-9673-420b-bd0c-2b2fca6312fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:04.055901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-3097a1a6-9673-420b-bd0c-2b2fca6312fc None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 09:00:04.057309 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-3097a1a6-9673-420b-bd0c-2b2fca6312fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:04.065940 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1447/2881] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 09:00:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 09:00:04.070351 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff67f43-1fa4-4e60-a6ed-a9d6c8351d65 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:04.080422 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff67f43-1fa4-4e60-a6ed-a9d6c8351d65 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:04.085134 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329b231-7cb6-49fd-9c24-876ee21a3f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:04.085805 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329b231-7cb6-49fd-9c24-876ee21a3f6f admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:04.086093 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329b231-7cb6-49fd-9c24-876ee21a3f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:04.086371 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329b231-7cb6-49fd-9c24-876ee21a3f6f admin admin] PATH_INFO: `/v3/domains` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:04.087194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329b231-7cb6-49fd-9c24-876ee21a3f6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86751) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 09:00:04.087439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329b231-7cb6-49fd-9c24-876ee21a3f6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:04.089149 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329b231-7cb6-49fd-9c24-876ee21a3f6f admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:04.101873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1435/2882] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Jul 25 09:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 09:00:04.106288 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097a1a6-9673-420b-bd0c-2b2fca6312fc None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:04.116413 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097a1a6-9673-420b-bd0c-2b2fca6312fc None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:04.121509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187176b0-2d29-44d5-8797-974345ee8085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:04.122173 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187176b0-2d29-44d5-8797-974345ee8085 admin admin] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:04.122455 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187176b0-2d29-44d5-8797-974345ee8085 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:04.122734 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187176b0-2d29-44d5-8797-974345ee8085 admin admin] PATH_INFO: `/v3/projects` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:04.123429 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187176b0-2d29-44d5-8797-974345ee8085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:04.125370 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187176b0-2d29-44d5-8797-974345ee8085 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:04.152983 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1448/2883] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Jul 25 09:00:04 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 09:00:04.158401 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329b231-7cb6-49fd-9c24-876ee21a3f6f admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:04.169166 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329b231-7cb6-49fd-9c24-876ee21a3f6f admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:04.233726 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22422fb-ec68-4db3-9c45-366d61941999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:04.234432 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22422fb-ec68-4db3-9c45-366d61941999 admin admin] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:04.234575 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22422fb-ec68-4db3-9c45-366d61941999 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:04.234850 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22422fb-ec68-4db3-9c45-366d61941999 admin admin] PATH_INFO: `/v3/users` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:04.235555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22422fb-ec68-4db3-9c45-366d61941999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:04.238130 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22422fb-ec68-4db3-9c45-366d61941999 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:04.260453 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1436/2884] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Jul 25 09:00:04 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 09:00:04.264534 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187176b0-2d29-44d5-8797-974345ee8085 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:04.275122 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187176b0-2d29-44d5-8797-974345ee8085 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:04.283163 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0b98b-b7e6-4f36-b7ea-7b3b6b3458d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:04.284371 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0b98b-b7e6-4f36-b7ea-7b3b6b3458d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:04.284702 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0b98b-b7e6-4f36-b7ea-7b3b6b3458d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:04.284987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0b98b-b7e6-4f36-b7ea-7b3b6b3458d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:04.285722 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0b98b-b7e6-4f36-b7ea-7b3b6b3458d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:04.287334 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0b98b-b7e6-4f36-b7ea-7b3b6b3458d8 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:04.298590 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1449/2885] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 09:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 09:00:04.303234 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22422fb-ec68-4db3-9c45-366d61941999 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:04.313909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22422fb-ec68-4db3-9c45-366d61941999 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:04.319006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584af44-a366-4f5a-819f-b81dfba13710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:04.319686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584af44-a366-4f5a-819f-b81dfba13710 admin admin] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:04.319968 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584af44-a366-4f5a-819f-b81dfba13710 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:04.320241 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584af44-a366-4f5a-819f-b81dfba13710 admin admin] PATH_INFO: `/v3/roles` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:04.321041 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584af44-a366-4f5a-819f-b81dfba13710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:04.322647 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584af44-a366-4f5a-819f-b81dfba13710 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:04.333430 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1437/2886] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Jul 25 09:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 09:00:04.337750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0b98b-b7e6-4f36-b7ea-7b3b6b3458d8 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:04.350559 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0b98b-b7e6-4f36-b7ea-7b3b6b3458d8 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:04.355430 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a812667-2d95-44f7-95dd-34cb132e4a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:04.356283 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a812667-2d95-44f7-95dd-34cb132e4a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:04.356586 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a812667-2d95-44f7-95dd-34cb132e4a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:04.356901 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a812667-2d95-44f7-95dd-34cb132e4a55 admin admin] PATH_INFO: `/v3/projects/8ced335a2fde4b6eb67f8a141fb7bc16/users/99d56db4f975434da59a1af867b64bd2/roles/82ee29c384454b9aa4db085c9d2e21e8` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:04.373239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a812667-2d95-44f7-95dd-34cb132e4a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ced335a2fde4b6eb67f8a141fb7bc16, user_id=99d56db4f975434da59a1af867b64bd2, role_id=82ee29c384454b9aa4db085c9d2e21e8)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:04.375985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a812667-2d95-44f7-95dd-34cb132e4a55 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:04.392352 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1450/2887] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Jul 25 09:00:04 2022] PUT /identity/v3/projects/8ced335a2fde4b6eb67f8a141fb7bc16/users/99d56db4f975434da59a1af867b64bd2/roles/82ee29c384454b9aa4db085c9d2e21e8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:00:04.402885 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824bf9f-7c53-4902-ba14-f50384b7bf78 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:04.403176 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824bf9f-7c53-4902-ba14-f50384b7bf78 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:04.403462 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824bf9f-7c53-4902-ba14-f50384b7bf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:04.434461 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-c824bf9f-7c53-4902-ba14-f50384b7bf78 None None] MFA Rules not processed for user `99d56db4f975434da59a1af867b64bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 09:00:04.435721 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-c824bf9f-7c53-4902-ba14-f50384b7bf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:04.507301 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1438/2888] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 09:00:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 09:00:04.522344 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a812667-2d95-44f7-95dd-34cb132e4a55 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:04.524208 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-2a812667-2d95-44f7-95dd-34cb132e4a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:04.585750 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a812667-2d95-44f7-95dd-34cb132e4a55 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:04.589796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853bd17-d64d-4981-9c9f-c153f917784e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:04.590466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853bd17-d64d-4981-9c9f-c153f917784e service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:04.590763 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853bd17-d64d-4981-9c9f-c153f917784e service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:04.591106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853bd17-d64d-4981-9c9f-c153f917784e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:04.601107 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853bd17-d64d-4981-9c9f-c153f917784e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:04.602561 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853bd17-d64d-4981-9c9f-c153f917784e service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:04.613915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1451/2889] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:00:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:00:14.503961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:00:14.607774 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:00:27.044443 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824bf9f-7c53-4902-ba14-f50384b7bf78 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:27.105251 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824bf9f-7c53-4902-ba14-f50384b7bf78 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:27.109802 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:27.110668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:27.110949 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:27.111220 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] PATH_INFO: `/v3/users/99d56db4f975434da59a1af867b64bd2` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:27.112402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d56db4f975434da59a1af867b64bd2)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:27.114835 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:27.158373 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d56db4f975434da59a1af867b64bd2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:27.165343 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d56db4f975434da59a1af867b64bd2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:27.177649 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d56db4f975434da59a1af867b64bd2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:27.184364 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d56db4f975434da59a1af867b64bd2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:27.185027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d56db4f975434da59a1af867b64bd2'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:27.190877 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1439/2890] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 09:00:27 2022] DELETE /identity/v3/users/99d56db4f975434da59a1af867b64bd2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 09:00:27.196635 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8caee-01fc-4f81-9995-90be19d12545 None None] REQUEST_METHOD: `POST` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:27.197121 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8caee-01fc-4f81-9995-90be19d12545 None None] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:27.197243 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8caee-01fc-4f81-9995-90be19d12545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:27.216061 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.auth.core [None req-def8caee-01fc-4f81-9995-90be19d12545 None None] MFA Rules not processed for user `fcb4950bc93048a891412be785f132b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86752) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 09:00:27.217299 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-def8caee-01fc-4f81-9995-90be19d12545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:27.278740 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1452/2891] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 09:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 09:00:27.288336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:27.289833 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:27.346997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311b658-be8b-455f-aff1-9027a414d8c3 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:27.350715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c601-0043-4c15-847f-0a160a99129e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:27.351365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c601-0043-4c15-847f-0a160a99129e service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:27.351660 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c601-0043-4c15-847f-0a160a99129e service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:27.351940 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c601-0043-4c15-847f-0a160a99129e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:27.361049 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1c601-0043-4c15-847f-0a160a99129e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:27.362488 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1c601-0043-4c15-847f-0a160a99129e service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:27.372690 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1440/2892] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:00:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:00:27.904480 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8caee-01fc-4f81-9995-90be19d12545 None None] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:00:27.906069 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-def8caee-01fc-4f81-9995-90be19d12545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:00:27.920801 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8caee-01fc-4f81-9995-90be19d12545 None None] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:00:27.925437 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:00:27.927084 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:00:27.927402 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:00:27.927775 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] PATH_INFO: `/v3/projects/8ced335a2fde4b6eb67f8a141fb7bc16` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:00:27.928989 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ced335a2fde4b6eb67f8a141fb7bc16)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:00:27.930772 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:00:27.978539 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ced335a2fde4b6eb67f8a141fb7bc16', 'request_id': 'req-de854014-9eda-4333-967a-1134f61a6e24'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:27.985355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ced335a2fde4b6eb67f8a141fb7bc16', 'request_id': 'req-de854014-9eda-4333-967a-1134f61a6e24'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:27.990806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ced335a2fde4b6eb67f8a141fb7bc16', 'request_id': 'req-de854014-9eda-4333-967a-1134f61a6e24'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:28.002055 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] The token cache is being invalidate because project 8ced335a2fde4b6eb67f8a141fb7bc16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 09:00:28.002339 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:28.002991 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:00:28.004548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1453/2893] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 09:00:27 2022] DELETE /identity/v3/projects/8ced335a2fde4b6eb67f8a141fb7bc16 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:00:37.368236 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:00:37.999109 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:01:01.868306 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c601-0043-4c15-847f-0a160a99129e service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:01:01.869947 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a2b1c601-0043-4c15-847f-0a160a99129e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:01:01.882019 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c601-0043-4c15-847f-0a160a99129e service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:01:01.929838 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035274e-0244-4c46-9663-315aa5b5c5cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:01:01.930486 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035274e-0244-4c46-9663-315aa5b5c5cd service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:01:01.930770 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035274e-0244-4c46-9663-315aa5b5c5cd service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:01:01.931047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035274e-0244-4c46-9663-315aa5b5c5cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:01:01.932859 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-6035274e-0244-4c46-9663-315aa5b5c5cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:01:02.001979 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035274e-0244-4c46-9663-315aa5b5c5cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:01:02.003345 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035274e-0244-4c46-9663-315aa5b5c5cd service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:01:02.015119 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1441/2894] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:01:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:01:03.191328 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:01:03.192813 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:01:03.206806 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de854014-9eda-4333-967a-1134f61a6e24 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:01:03.210880 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a65e1f-e217-4b3d-ada5-3699150265b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:01:03.211514 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a65e1f-e217-4b3d-ada5-3699150265b3 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:01:03.211796 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a65e1f-e217-4b3d-ada5-3699150265b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:01:03.212070 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a65e1f-e217-4b3d-ada5-3699150265b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:01:03.213791 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-31a65e1f-e217-4b3d-ada5-3699150265b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:01:03.227594 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a65e1f-e217-4b3d-ada5-3699150265b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:01:03.229058 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a65e1f-e217-4b3d-ada5-3699150265b3 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:01:03.240092 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1454/2895] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:01:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:01:05.543480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd52af-343c-4927-8fc3-212c26ca7fa4 None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:01:05.544081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd52af-343c-4927-8fc3-212c26ca7fa4 None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:01:05.544081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd52af-343c-4927-8fc3-212c26ca7fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:01:05.564047 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-2afd52af-343c-4927-8fc3-212c26ca7fa4 None None] MFA Rules not processed for user `a1a3d224feb94a1baa75a03bc8d1132a`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 09:01:05.565291 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2afd52af-343c-4927-8fc3-212c26ca7fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:01:05.628695 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1442/2896] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 09:01:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 09:01:13.235035 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=6 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:01:15.616902 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=79 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:01:52.665468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a65e1f-e217-4b3d-ada5-3699150265b3 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:01:52.675181 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a65e1f-e217-4b3d-ada5-3699150265b3 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:01:52.678977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2bd86-c0de-42c7-ab29-ff56f1fa2922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:01:52.679616 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2bd86-c0de-42c7-ab29-ff56f1fa2922 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:01:52.679897 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2bd86-c0de-42c7-ab29-ff56f1fa2922 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:01:52.680178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2bd86-c0de-42c7-ab29-ff56f1fa2922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:01:52.682015 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7cc2bd86-c0de-42c7-ab29-ff56f1fa2922 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:01:52.692095 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2bd86-c0de-42c7-ab29-ff56f1fa2922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:01:52.693548 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2bd86-c0de-42c7-ab29-ff56f1fa2922 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:01:52.705073 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1455/2897] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:01:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:02:02.699418 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=6 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:02:40.782332 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd52af-343c-4927-8fc3-212c26ca7fa4 None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:02:40.792753 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd52af-343c-4927-8fc3-212c26ca7fa4 None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:02:40.796322 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783196f-6630-4fe1-a73a-6f9b945b9375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:02:40.797001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783196f-6630-4fe1-a73a-6f9b945b9375 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:02:40.797286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783196f-6630-4fe1-a73a-6f9b945b9375 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:02:40.797565 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783196f-6630-4fe1-a73a-6f9b945b9375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:02:40.799336 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-2783196f-6630-4fe1-a73a-6f9b945b9375 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:02:40.809409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783196f-6630-4fe1-a73a-6f9b945b9375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:02:40.810819 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783196f-6630-4fe1-a73a-6f9b945b9375 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:02:40.819867 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1443/2898] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:02:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:02:40.913757 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2bd86-c0de-42c7-ab29-ff56f1fa2922 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:02:40.915258 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-7cc2bd86-c0de-42c7-ab29-ff56f1fa2922 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:02:40.927129 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2bd86-c0de-42c7-ab29-ff56f1fa2922 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:02:40.930923 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bab84c-d186-4d09-88a2-f7e842292fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:02:40.932178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bab84c-d186-4d09-88a2-f7e842292fdc service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:02:40.932468 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bab84c-d186-4d09-88a2-f7e842292fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:02:40.932784 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bab84c-d186-4d09-88a2-f7e842292fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:02:40.934631 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-56bab84c-d186-4d09-88a2-f7e842292fdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:02:40.944490 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bab84c-d186-4d09-88a2-f7e842292fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:02:40.945964 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bab84c-d186-4d09-88a2-f7e842292fdc service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:02:40.954205 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1456/2899] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:02:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:02:41.029985 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783196f-6630-4fe1-a73a-6f9b945b9375 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:02:41.039531 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783196f-6630-4fe1-a73a-6f9b945b9375 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:02:41.047256 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f8bf1-5249-45c0-ac0a-b45d50b5b5fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:02:41.048218 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f8bf1-5249-45c0-ac0a-b45d50b5b5fa service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:02:41.048739 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f8bf1-5249-45c0-ac0a-b45d50b5b5fa service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:02:41.049179 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f8bf1-5249-45c0-ac0a-b45d50b5b5fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:02:41.051554 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e01f8bf1-5249-45c0-ac0a-b45d50b5b5fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:02:41.067854 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f8bf1-5249-45c0-ac0a-b45d50b5b5fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:02:41.069316 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f8bf1-5249-45c0-ac0a-b45d50b5b5fa service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:02:41.081740 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1444/2900] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:02:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:02:41.146509 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bab84c-d186-4d09-88a2-f7e842292fdc service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:02:41.147983 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-56bab84c-d186-4d09-88a2-f7e842292fdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:02:41.221891 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bab84c-d186-4d09-88a2-f7e842292fdc service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:02:41.227418 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d8717-3f0b-43c8-8a13-4575861a4657 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:02:41.228948 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d8717-3f0b-43c8-8a13-4575861a4657 service placement] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:02:41.229336 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d8717-3f0b-43c8-8a13-4575861a4657 service placement] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:02:41.229624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d8717-3f0b-43c8-8a13-4575861a4657 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:02:41.231466 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-173d8717-3f0b-43c8-8a13-4575861a4657 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:02:41.243994 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d8717-3f0b-43c8-8a13-4575861a4657 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:02:41.245927 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d8717-3f0b-43c8-8a13-4575861a4657 service placement] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:02:41.257151 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1457/2901] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:02:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:02:41.311153 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f8bf1-5249-45c0-ac0a-b45d50b5b5fa service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:02:41.314073 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e01f8bf1-5249-45c0-ac0a-b45d50b5b5fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:02:41.328994 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f8bf1-5249-45c0-ac0a-b45d50b5b5fa service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:02:41.334997 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0311399-d6f7-452a-be76-00482d0afad5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:02:41.335675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0311399-d6f7-452a-be76-00482d0afad5 service nova] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:02:41.335956 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0311399-d6f7-452a-be76-00482d0afad5 service nova] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:02:41.336229 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0311399-d6f7-452a-be76-00482d0afad5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:02:41.338552 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-a0311399-d6f7-452a-be76-00482d0afad5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:02:41.353220 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0311399-d6f7-452a-be76-00482d0afad5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:02:41.355421 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0311399-d6f7-452a-be76-00482d0afad5 service nova] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:02:41.377938 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1445/2902] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:02:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:02:51.252369 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=38 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:02:51.363508 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=18 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:03:13.645959 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d8717-3f0b-43c8-8a13-4575861a4657 service placement] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:03:13.647358 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-173d8717-3f0b-43c8-8a13-4575861a4657 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:03:13.704253 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d8717-3f0b-43c8-8a13-4575861a4657 service placement] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:03:13.708128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4c043-64eb-4f4f-b152-dd857503920b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84871b711bcb47efb039ce40766cf3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:03:13.709021 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4c043-64eb-4f4f-b152-dd857503920b service cinder] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:03:13.709206 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4c043-64eb-4f4f-b152-dd857503920b service cinder] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:03:13.709344 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4c043-64eb-4f4f-b152-dd857503920b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:03:13.711103 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-5fb4c043-64eb-4f4f-b152-dd857503920b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:03:13.722408 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4c043-64eb-4f4f-b152-dd857503920b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:03:13.723885 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4c043-64eb-4f4f-b152-dd857503920b service cinder] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:03:13.733495 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1458/2903] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:03:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:03:23.728846 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=26 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:03:24.767190 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf75e91-af72-4a89-8189-77bf43eb800f None None] REQUEST_METHOD: `POST` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:03:24.767751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf75e91-af72-4a89-8189-77bf43eb800f None None] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:03:24.767869 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf75e91-af72-4a89-8189-77bf43eb800f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:03:24.788652 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.auth.core [None req-eaf75e91-af72-4a89-8189-77bf43eb800f None None] MFA Rules not processed for user `a1a3d224feb94a1baa75a03bc8d1132a`. Rule list: `[]` (Enabled: `True`). {{(pid=86751) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 09:03:24.789909 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-eaf75e91-af72-4a89-8189-77bf43eb800f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:03:24.798077 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1446/2904] 10.222.0.52 () {58 vars in 958 bytes} [Mon Jul 25 09:03:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 09:03:34.783706 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=3 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:03:42.235461 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4c043-64eb-4f4f-b152-dd857503920b service cinder] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:03:42.245052 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4c043-64eb-4f4f-b152-dd857503920b service cinder] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:03:42.248662 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164dc88-c3b0-4f9f-900e-d451eb5644fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:03:42.249298 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164dc88-c3b0-4f9f-900e-d451eb5644fb service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:03:42.249579 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164dc88-c3b0-4f9f-900e-d451eb5644fb service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:03:42.249856 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164dc88-c3b0-4f9f-900e-d451eb5644fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:03:42.251598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d164dc88-c3b0-4f9f-900e-d451eb5644fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:03:42.313578 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164dc88-c3b0-4f9f-900e-d451eb5644fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:03:42.315020 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164dc88-c3b0-4f9f-900e-d451eb5644fb service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:03:42.325128 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1459/2905] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:03:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:03:52.320751 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=30 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:04:41.035036 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf75e91-af72-4a89-8189-77bf43eb800f None None] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:04:41.056196 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf75e91-af72-4a89-8189-77bf43eb800f None None] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:04:41.060001 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4363a-3f72-4f4a-838b-35200890cc05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:04:41.060686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4363a-3f72-4f4a-838b-35200890cc05 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:04:41.060971 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4363a-3f72-4f4a-838b-35200890cc05 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:04:41.061244 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4363a-3f72-4f4a-838b-35200890cc05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:04:41.063000 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-84c4363a-3f72-4f4a-838b-35200890cc05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:04:41.147491 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4363a-3f72-4f4a-838b-35200890cc05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:04:41.148963 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4363a-3f72-4f4a-838b-35200890cc05 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:04:41.157961 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1447/2906] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:04:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:04:41.997426 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164dc88-c3b0-4f9f-900e-d451eb5644fb service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:04:41.999251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d164dc88-c3b0-4f9f-900e-d451eb5644fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:04:42.118232 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164dc88-c3b0-4f9f-900e-d451eb5644fb service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:04:42.126808 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2e738-8253-42c5-b07e-e8a971652878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:04:42.127481 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2e738-8253-42c5-b07e-e8a971652878 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:04:42.127768 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2e738-8253-42c5-b07e-e8a971652878 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:04:42.128050 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2e738-8253-42c5-b07e-e8a971652878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:04:42.130239 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-d8b2e738-8253-42c5-b07e-e8a971652878 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:04:42.146296 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2e738-8253-42c5-b07e-e8a971652878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:04:42.147952 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2e738-8253-42c5-b07e-e8a971652878 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:04:42.159996 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1460/2907] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:04:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:04:45.320006 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4363a-3f72-4f4a-838b-35200890cc05 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:04:45.342668 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4363a-3f72-4f4a-838b-35200890cc05 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:04:45.347972 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69efe1c-629c-475b-bf13-f44f69ce6c94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:04:45.348686 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69efe1c-629c-475b-bf13-f44f69ce6c94 service placement] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:04:45.348975 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69efe1c-629c-475b-bf13-f44f69ce6c94 service placement] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:04:45.349258 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69efe1c-629c-475b-bf13-f44f69ce6c94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:04:45.351110 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b69efe1c-629c-475b-bf13-f44f69ce6c94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:04:45.378641 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69efe1c-629c-475b-bf13-f44f69ce6c94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:04:45.380257 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69efe1c-629c-475b-bf13-f44f69ce6c94 service placement] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:04:45.391550 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1448/2908] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:04:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:04:47.070606 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2e738-8253-42c5-b07e-e8a971652878 service nova] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:04:47.080432 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2e738-8253-42c5-b07e-e8a971652878 service nova] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:04:47.084101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4d0ce-a060-4de5-a25c-22e7f47a3472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:04:47.085399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4d0ce-a060-4de5-a25c-22e7f47a3472 service neutron] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:04:47.085790 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4d0ce-a060-4de5-a25c-22e7f47a3472 service neutron] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:04:47.086081 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4d0ce-a060-4de5-a25c-22e7f47a3472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:04:47.087980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f2e4d0ce-a060-4de5-a25c-22e7f47a3472 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:04:47.160251 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4d0ce-a060-4de5-a25c-22e7f47a3472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:04:47.161739 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4d0ce-a060-4de5-a25c-22e7f47a3472 service neutron] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:04:47.171837 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1461/2909] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:04:50.312286 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69efe1c-629c-475b-bf13-f44f69ce6c94 service placement] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:04:50.313830 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-b69efe1c-629c-475b-bf13-f44f69ce6c94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:04:50.326951 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69efe1c-629c-475b-bf13-f44f69ce6c94 service placement] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:04:50.332243 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6845e-68cf-4d17-8212-1f00dca68f1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:04:50.333035 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6845e-68cf-4d17-8212-1f00dca68f1c service placement] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:04:50.333386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6845e-68cf-4d17-8212-1f00dca68f1c service placement] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:04:50.333730 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6845e-68cf-4d17-8212-1f00dca68f1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:04:50.336110 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-e6a6845e-68cf-4d17-8212-1f00dca68f1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:04:50.348334 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6845e-68cf-4d17-8212-1f00dca68f1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:04:50.350715 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6845e-68cf-4d17-8212-1f00dca68f1c service placement] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:04:50.362315 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1449/2910] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:04:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:04:53.230760 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4d0ce-a060-4de5-a25c-22e7f47a3472 service neutron] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:04:53.232223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-f2e4d0ce-a060-4de5-a25c-22e7f47a3472 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:04:53.281025 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4d0ce-a060-4de5-a25c-22e7f47a3472 service neutron] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:04:53.285657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:04:53.286537 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:04:53.287657 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:04:53.287977 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] PATH_INFO: `/v3/users/52e9d0e3c2dd453eb4bfc2db6b45d80f` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:04:53.289263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=52e9d0e3c2dd453eb4bfc2db6b45d80f)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:04:53.291269 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:04:53.329018 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:04:53.342417 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52e9d0e3c2dd453eb4bfc2db6b45d80f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.343068 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52e9d0e3c2dd453eb4bfc2db6b45d80f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.350604 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52e9d0e3c2dd453eb4bfc2db6b45d80f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.367271 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52e9d0e3c2dd453eb4bfc2db6b45d80f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.374385 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52e9d0e3c2dd453eb4bfc2db6b45d80f'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.380368 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1462/2911] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 09:04:53 2022] DELETE /identity/v3/users/52e9d0e3c2dd453eb4bfc2db6b45d80f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:04:53.558386 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6845e-68cf-4d17-8212-1f00dca68f1c service placement] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:04:53.570911 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6845e-68cf-4d17-8212-1f00dca68f1c service placement] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:04:53.632102 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:04:53.633148 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:04:53.633429 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:04:53.633696 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] PATH_INFO: `/v3/projects/4a77723e8867459c99fcd86914fa44df` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:04:53.634882 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a77723e8867459c99fcd86914fa44df)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:04:53.636810 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:04:53.661137 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=100 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:04:53.687983 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a77723e8867459c99fcd86914fa44df', 'request_id': 'req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.696650 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a77723e8867459c99fcd86914fa44df', 'request_id': 'req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.703013 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a77723e8867459c99fcd86914fa44df', 'request_id': 'req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.715396 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] The token cache is being invalidate because project 4a77723e8867459c99fcd86914fa44df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 09:04:53.715685 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.716330 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.718379 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1450/2912] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 09:04:53 2022] DELETE /identity/v3/projects/4a77723e8867459c99fcd86914fa44df => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:04:53.722427 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:04:53.724140 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:04:53.737879 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e09e3-0aaf-431d-9dbb-f1ce7ca96ebd admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:04:53.794882 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:04:53.795799 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:04:53.796086 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:04:53.796362 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] PATH_INFO: `/v3/users/aa15c710c8ad42119203b1722750bf85` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:04:53.797641 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa15c710c8ad42119203b1722750bf85)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:04:53.799582 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:04:53.851625 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa15c710c8ad42119203b1722750bf85'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.852282 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa15c710c8ad42119203b1722750bf85'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.859987 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa15c710c8ad42119203b1722750bf85'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.873661 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa15c710c8ad42119203b1722750bf85'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.880719 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa15c710c8ad42119203b1722750bf85'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:53.886812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1463/2913] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 09:04:53 2022] DELETE /identity/v3/users/aa15c710c8ad42119203b1722750bf85 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:04:54.069313 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:04:54.070873 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:04:54.091063 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c3c9e-3c23-48b8-b30c-d6bb1e5980bb admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:04:54.151402 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:04:54.152289 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:04:54.152600 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:04:54.152876 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] PATH_INFO: `/v3/projects/9b26a9a5df2b447cae8fc546e3e252c6` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:04:54.154081 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b26a9a5df2b447cae8fc546e3e252c6)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:04:54.156363 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:04:54.207206 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b26a9a5df2b447cae8fc546e3e252c6', 'request_id': 'req-949797cc-0e86-48a4-9bcf-dcaec3229a57'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:54.215290 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b26a9a5df2b447cae8fc546e3e252c6', 'request_id': 'req-949797cc-0e86-48a4-9bcf-dcaec3229a57'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:54.221040 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b26a9a5df2b447cae8fc546e3e252c6', 'request_id': 'req-949797cc-0e86-48a4-9bcf-dcaec3229a57'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:54.233184 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] The token cache is being invalidate because project 9b26a9a5df2b447cae8fc546e3e252c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86751) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 09:04:54.233473 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:54.234129 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:04:54.235694 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1451/2914] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 09:04:54 2022] DELETE /identity/v3/projects/9b26a9a5df2b447cae8fc546e3e252c6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:05:03.883349 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=70,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:05:04.230051 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:05:44.943812 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:05:44.946899 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:05:45.030985 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e2c91-55f6-46cc-be9c-4f56fa2dd7a7 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:05:45.120445 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74252f28-757a-4982-9d97-c6112fe5ab27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b476d6b82ae464b81fc8498dc0e68c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:05:45.121178 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252f28-757a-4982-9d97-c6112fe5ab27 service nova] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:05:45.121471 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252f28-757a-4982-9d97-c6112fe5ab27 service nova] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:05:45.121752 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252f28-757a-4982-9d97-c6112fe5ab27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:05:45.123682 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-74252f28-757a-4982-9d97-c6112fe5ab27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:05:45.137624 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74252f28-757a-4982-9d97-c6112fe5ab27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:05:45.139248 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74252f28-757a-4982-9d97-c6112fe5ab27 service nova] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:05:45.151562 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1464/2915] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:05:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:05:55.146753 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=62 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:05:57.639789 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:05:57.642027 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:05:57.720870 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949797cc-0e86-48a4-9bcf-dcaec3229a57 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:05:57.724504 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9f865-7427-4e5b-91ff-6d52b27730b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62688bc3d8a449c5bc9e0d64cd2462b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:05:57.725185 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9f865-7427-4e5b-91ff-6d52b27730b8 service neutron] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:05:57.725472 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9f865-7427-4e5b-91ff-6d52b27730b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:05:57.725751 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9f865-7427-4e5b-91ff-6d52b27730b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:05:57.727519 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-aac9f865-7427-4e5b-91ff-6d52b27730b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:05:57.790555 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9f865-7427-4e5b-91ff-6d52b27730b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:05:57.791989 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9f865-7427-4e5b-91ff-6d52b27730b8 service neutron] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:05:57.802092 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1452/2916] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 09:06:03.922598 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74252f28-757a-4982-9d97-c6112fe5ab27 service nova] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:06:03.924034 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-74252f28-757a-4982-9d97-c6112fe5ab27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:06:03.980008 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9f865-7427-4e5b-91ff-6d52b27730b8 service neutron] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:06:03.981659 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-aac9f865-7427-4e5b-91ff-6d52b27730b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:06:03.994250 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9f865-7427-4e5b-91ff-6d52b27730b8 service neutron] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:06:03.999683 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:06:04.000529 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:06:04.000843 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:06:04.001115 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] PATH_INFO: `/v3/users/a1a3d224feb94a1baa75a03bc8d1132a` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:06:04.002308 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a3d224feb94a1baa75a03bc8d1132a)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:06:04.002515 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74252f28-757a-4982-9d97-c6112fe5ab27 service nova] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:06:04.004220 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:06:04.007595 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f9844-882c-4724-9d5a-43c0f70af114 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:06:04.008248 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f9844-882c-4724-9d5a-43c0f70af114 service placement] REQUEST_METHOD: `GET` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:06:04.008527 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f9844-882c-4724-9d5a-43c0f70af114 service placement] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:06:04.008832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f9844-882c-4724-9d5a-43c0f70af114 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:06:04.010640 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-b82f9844-882c-4724-9d5a-43c0f70af114 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:06:04.023106 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f9844-882c-4724-9d5a-43c0f70af114 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:06:04.024713 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f9844-882c-4724-9d5a-43c0f70af114 service placement] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:06:04.035859 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1465/2917] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:06:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:06:04.057020 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a3d224feb94a1baa75a03bc8d1132a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.064617 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a3d224feb94a1baa75a03bc8d1132a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.077165 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a3d224feb94a1baa75a03bc8d1132a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.084497 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a3d224feb94a1baa75a03bc8d1132a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.085194 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a3d224feb94a1baa75a03bc8d1132a'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.091168 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1453/2918] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 09:06:03 2022] DELETE /identity/v3/users/a1a3d224feb94a1baa75a03bc8d1132a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:06:04.262276 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f9844-882c-4724-9d5a-43c0f70af114 service placement] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:06:04.274315 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f9844-882c-4724-9d5a-43c0f70af114 service placement] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:06:04.336457 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:06:04.337355 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:06:04.337639 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:06:04.337915 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] PATH_INFO: `/v3/projects/5005a107ff1940908069356c788185f6` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:06:04.339104 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5005a107ff1940908069356c788185f6)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:06:04.341040 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:06:04.387549 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5005a107ff1940908069356c788185f6', 'request_id': 'req-fb29909a-eadb-4ecc-8558-117a081aa7a6'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.395488 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5005a107ff1940908069356c788185f6', 'request_id': 'req-fb29909a-eadb-4ecc-8558-117a081aa7a6'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.401092 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5005a107ff1940908069356c788185f6', 'request_id': 'req-fb29909a-eadb-4ecc-8558-117a081aa7a6'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.411973 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] The token cache is being invalidate because project 5005a107ff1940908069356c788185f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 09:06:04.412263 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.412922 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.414472 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1466/2919] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 09:06:04 2022] DELETE /identity/v3/projects/5005a107ff1940908069356c788185f6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:06:04.418480 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:06:04.419865 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:06:04.428007 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:06:04.433933 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cfdb5-b29e-46a6-9464-10dd7f9e2026 admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:06:04.490439 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:06:04.491320 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:06:04.491603 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:06:04.491878 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] PATH_INFO: `/v3/users/c4f250476ffc4958aafa960ef85d6546` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:06:04.493365 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f250476ffc4958aafa960ef85d6546)` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:06:04.495275 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:06:04.546409 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f250476ffc4958aafa960ef85d6546'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.553964 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f250476ffc4958aafa960ef85d6546'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.566280 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f250476ffc4958aafa960ef85d6546'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.573284 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f250476ffc4958aafa960ef85d6546'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.573910 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.notifications [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f250476ffc4958aafa960ef85d6546'} {{(pid=86751) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.580182 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1454/2920] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Jul 25 09:06:04 2022] DELETE /identity/v3/users/c4f250476ffc4958aafa960ef85d6546 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:06:04.750980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] Authenticating user token {{(pid=86752) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:06:04.752497 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.fernet_utils [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86752) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:06:04.765699 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29909a-eadb-4ecc-8558-117a081aa7a6 admin admin] Validating token access rules against request {{(pid=86752) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:06:04.783101 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:06:04.825716 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb4950bc93048a891412be785f132b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86d2d432f134fa4aeaa99c3935e4c0c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86752) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:06:04.826513 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:06:04.826832 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:06:04.827114 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2 admin admin] PATH_INFO: `/v3/projects/c9b5e4cfcdee4d96a8b9f08f8ed323a1` {{(pid=86752) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:06:04.828399 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b5e4cfcdee4d96a8b9f08f8ed323a1)` {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:06:04.830343 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2 admin admin] RBAC: Authorization granted {{(pid=86752) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:06:04.877010 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b5e4cfcdee4d96a8b9f08f8ed323a1', 'request_id': 'req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.883980 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b5e4cfcdee4d96a8b9f08f8ed323a1', 'request_id': 'req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.889777 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b5e4cfcdee4d96a8b9f08f8ed323a1', 'request_id': 'req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2'} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.900932 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2 admin admin] The token cache is being invalidate because project c9b5e4cfcdee4d96a8b9f08f8ed323a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86752) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 09:06:04.901223 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.901860 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG keystone.notifications [None req-e8a67753-b91d-4a7c-bdd6-ced71cdd5fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86752) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 09:06:04.903434 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: [pid: 86752|app: 0|req: 1467/2921] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Jul 25 09:06:04 2022] DELETE /identity/v3/projects/c9b5e4cfcdee4d96a8b9f08f8ed323a1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 09:06:14.576675 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=52,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:06:14.897712 dv-cinder-850840-1-iscsi devstack@keystone.service[86752]: DEBUG dbcounter [-] [86752] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86752) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 09:10:03.912389 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] Authenticating user token {{(pid=86751) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 09:10:03.913838 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:10:03.983265 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8b73d-bafd-4fde-8634-2c185a549aef admin admin] Validating token access rules against request {{(pid=86751) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 09:10:04.042672 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197177b3-4048-4970-8087-29dfaf34474e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62bf4ef4fce74f7b91e453f73430d9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ee27c4da77c47428e02fd65e890d256', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86751) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 09:10:04.043281 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197177b3-4048-4970-8087-29dfaf34474e service placement] REQUEST_METHOD: `GET` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 09:10:04.043569 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197177b3-4048-4970-8087-29dfaf34474e service placement] SCRIPT_NAME: `/identity` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 09:10:04.043847 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197177b3-4048-4970-8087-29dfaf34474e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86751) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 09:10:04.046165 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.fernet_utils [None req-197177b3-4048-4970-8087-29dfaf34474e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86751) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 09:10:04.057484 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197177b3-4048-4970-8087-29dfaf34474e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 09:10:04.059156 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197177b3-4048-4970-8087-29dfaf34474e service placement] RBAC: Authorization granted {{(pid=86751) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 09:10:04.070471 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: [pid: 86751|app: 0|req: 1455/2922] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Jul 25 09:10:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 09:10:14.066029 dv-cinder-850840-1-iscsi devstack@keystone.service[86751]: DEBUG dbcounter [-] [86751] Writing DB stats keystone:SELECT=58 {{(pid=86751) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}